2019-11-25T08:56:18.065 INFO:root:teuthology version: 1.0.0-c5aab4e 2019-11-25T08:56:18.066 DEBUG:teuthology.run:Teuthology command: teuthology /tmp/teuthology-worker.l8hKPb.tmp -- --verbose --lock --description rgw:verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} --name yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi --block --owner scheduled_yuvalif@teuthology --archive /home/teuthworker/archive/yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi/4542041 2019-11-25T08:56:18.103 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-25T08:56:18.161 INFO:teuthology.run:Config: archive_path: /home/teuthworker/archive/yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi/4542041 branch: wip-yuval-add-kafka-security2 description: rgw:verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml} email: yuvalif@yahoo.com first_in_suite: false job_id: '4542041' kernel: kdb: true sha1: distro last_in_suite: false machine_type: smithi name: yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi nuke-on-error: true openstack: - volumes: count: 4 size: 10 os_type: centos overrides: admin_socket: branch: wip-yuval-add-kafka-security2 ceph: conf: client: debug rgw: 20 rgw crypt require ssl: false rgw crypt s3 kms backend: testing rgw crypt s3 kms encryption keys: testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= rgw lc debug interval: 10 global: mon client directed command retry: 5 ms inject socket failures: 5000 osd heartbeat grace: 40 osd_max_pg_log_entries: 10 osd_min_pg_log_entries: 10 mon: debug mon: 20 debug ms: 1 debug paxos: 20 mon osd crush smoke test: false osd: debug filestore: 20 debug journal: 20 debug ms: 1 debug osd: 25 osd objectstore: filestore osd shutdown pgref assert: true osd sloppy crc: true fs: xfs log-whitelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(OSD_SLOW_PING_TIME sha1: 12b9659be24ae0e3d791f80d81504ab2770c1343 valgrind: mds: - --tool=memcheck mon: - --tool=memcheck - --leak-check=full - --show-reachable=yes osd: - --tool=memcheck ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: osd default pool size: 2 osd: osd objectstore: filestore osd sloppy crc: true filestore: true fs: xfs install: ceph: flavor: notcmalloc sha1: 12b9659be24ae0e3d791f80d81504ab2770c1343 rgw: compression type: random ec-data-pool: true frontend: civetweb storage classes: LUKEWARM, FROZEN s3tests: slow_backend: true workunit: branch: wip-yuval-add-kafka-security2 sha1: 12b9659be24ae0e3d791f80d81504ab2770c1343 owner: scheduled_yuvalif@teuthology priority: 100 repo: https://github.com/ceph/ceph-ci.git roles: - - mon.a - mon.c - mgr.y - osd.0 - osd.1 - osd.2 - osd.3 - client.0 - - mon.b - mgr.x - osd.4 - osd.5 - osd.6 - osd.7 - client.1 sha1: 12b9659be24ae0e3d791f80d81504ab2770c1343 suite: rgw:verify suite_branch: wip-yuval-add-kafka-security2 suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-add-kafka-security2/qa suite_relpath: qa suite_repo: git://git.ceph.com/ceph-ci.git suite_sha1: 12b9659be24ae0e3d791f80d81504ab2770c1343 tasks: - install: flavor: notcmalloc - ceph: null - openssl_keys: null - rgw: client.0: valgrind: - --tool=memcheck - --max-threads=1024 - workunit: clients: client.0: - cls/test_cls_lock.sh - cls/test_cls_log.sh - cls/test_cls_refcount.sh - cls/test_cls_rgw.sh - rgw/test_rgw_gc_log.sh - ragweed: client.0: default-branch: ceph-master rgw_server: client.0 stages: prepare,check - s3tests-java: client.0: force-branch: master force-repo: https://github.com/ceph/java_s3tests.git - s3tests: client.0: force-branch: ceph-master rgw_server: client.0 - swift: client.0: rgw_server: client.0 teuthology_branch: master tube: smithi verbose: false worker_log: /home/teuthworker/archive/worker_logs/worker.smithi.9224 2019-11-25T08:56:18.161 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-add-kafka-security2/qa; will attempt to use it 2019-11-25T08:56:18.162 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-add-kafka-security2/qa/tasks 2019-11-25T08:56:18.162 INFO:teuthology.run_tasks:Running task internal.check_packages... 2019-11-25T08:56:18.164 INFO:teuthology.task.internal:Checking packages... 2019-11-25T08:56:18.177 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'notcmalloc' and ceph hash '12b9659be24ae0e3d791f80d81504ab2770c1343' 2019-11-25T08:56:18.177 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2019-11-25T08:56:18.177 INFO:teuthology.packaging:ref: None 2019-11-25T08:56:18.178 INFO:teuthology.packaging:tag: None 2019-11-25T08:56:18.178 INFO:teuthology.packaging:branch: wip-yuval-add-kafka-security2 2019-11-25T08:56:18.178 INFO:teuthology.packaging:sha1: 12b9659be24ae0e3d791f80d81504ab2770c1343 2019-11-25T08:56:18.178 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&ref=wip-yuval-add-kafka-security2 2019-11-25T08:56:18.350 INFO:teuthology.task.internal:Found packages for ceph version 15.0.0-7614.g12b9659 2019-11-25T08:56:18.351 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2019-11-25T08:56:18.361 INFO:teuthology.task.internal:no buildpackages task found 2019-11-25T08:56:18.362 INFO:teuthology.run_tasks:Running task internal.lock_machines... 2019-11-25T08:56:18.410 INFO:teuthology.task.internal.lock_machines:Locking machines... 2019-11-25T08:56:18.410 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-25T08:56:18.456 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:56:28.472 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:56:38.491 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:56:48.501 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:56:58.516 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:57:08.534 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:57:18.548 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:57:28.564 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:57:38.579 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:57:48.592 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:57:58.608 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:58:08.625 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:58:18.641 INFO:teuthology.task.internal.lock_machines:waiting for more smithi machines to be free (need 2 + 2, have 2)... 2019-11-25T08:58:28.664 DEBUG:teuthology.lock.ops:lock_many request: {'count': 2, 'locked_by': 'scheduled_yuvalif@teuthology', 'description': '/home/teuthworker/archive/yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi/4542041', 'machine_type': 'smithi'} 2019-11-25T08:58:28.680 DEBUG:teuthology.lock.ops:locked ubuntu@smithi081.front.sepia.ceph.com, ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T08:58:28.829 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-11-25T08:58:29.027 INFO:teuthology.provision.fog.smithi081:Scheduling deploy of centos 7.6 2019-11-25T08:58:29.033 INFO:teuthology.provision.fog.smithi153:Scheduling deploy of centos 7.6 2019-11-25T08:58:29.331 INFO:teuthology.orchestra.console:Power off smithi081 2019-11-25T08:58:29.331 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-11-25T08:58:29.391 INFO:teuthology.orchestra.console:Power off smithi153 2019-11-25T08:58:29.392 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power off 2019-11-25T08:58:29.422 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:29.452 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:33.494 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:33.522 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:37.624 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:37.652 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:41.756 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:41.789 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:41.989 INFO:teuthology.orchestra.console:Power off for smithi081 completed 2019-11-25T08:58:42.025 INFO:teuthology.orchestra.console:Power off for smithi153 completed 2019-11-25T08:58:42.091 INFO:teuthology.orchestra.console:Power on smithi081 2019-11-25T08:58:42.091 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-11-25T08:58:42.127 INFO:teuthology.orchestra.console:Power on smithi153 2019-11-25T08:58:42.127 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power on 2019-11-25T08:58:42.168 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:42.206 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:46.244 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:46.272 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:50.377 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi081.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:50.406 DEBUG:teuthology.orchestra.console:pexpect command: ipmitool -H smithi153.ipmi.sepia.ceph.com -I lanplus -U inktank -P ApGNXcA7 power status 2019-11-25T08:58:50.608 INFO:teuthology.orchestra.console:Power on for smithi081 completed 2019-11-25T08:58:50.678 INFO:teuthology.orchestra.console:Power on for smithi153 completed 2019-11-25T08:58:50.709 INFO:teuthology.provision.fog.smithi081:Waiting for deploy to finish 2019-11-25T08:58:50.780 INFO:teuthology.provision.fog.smithi153:Waiting for deploy to finish 2019-11-25T09:01:36.746 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:01:36.762 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:01:45.805 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:01:45.833 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:01:54.864 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:01:54.890 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:03.950 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:03.951 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:13.005 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:13.006 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:22.065 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:22.066 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:31.153 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:31.154 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:40.237 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:49.293 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:02:55.537 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:03:04.625 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:03:13.709 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:03:13.885 INFO:teuthology.orchestra.run.smithi153:> hostname 2019-11-25T09:03:14.353 INFO:teuthology.orchestra.run.smithi153.stdout:smithi068 2019-11-25T09:03:14.354 INFO:teuthology.orchestra.run.smithi153:> grep smithi068 /etc/hosts 2019-11-25T09:03:14.411 INFO:teuthology.orchestra.run.smithi153.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068 2019-11-25T09:03:14.412 INFO:teuthology.orchestra.run.smithi153:> sudo hostname smithi153 2019-11-25T09:03:14.530 INFO:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi068/smithi153/g' /etc/hosts 2019-11-25T09:03:14.621 INFO:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/172.21.15.68/172.21.15.153/g' /etc/hosts 2019-11-25T09:03:14.695 INFO:teuthology.orchestra.run.smithi153:> sudo sed -i -e 's/smithi068/smithi153/g' /etc/hostname 2019-11-25T09:03:14.757 INFO:teuthology.orchestra.run.smithi153:> sudo hostname smithi153 2019-11-25T09:03:14.818 INFO:teuthology.provision.fog.smithi153:Deploy complete! 2019-11-25T09:03:14.820 DEBUG:teuthology.parallel:result is None 2019-11-25T09:03:26.773 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:03:27.015 INFO:teuthology.orchestra.run.smithi081:> hostname 2019-11-25T09:03:27.468 INFO:teuthology.orchestra.run.smithi081.stdout:smithi068 2019-11-25T09:03:27.468 INFO:teuthology.orchestra.run.smithi081:> grep smithi068 /etc/hosts 2019-11-25T09:03:27.510 INFO:teuthology.orchestra.run.smithi081.stdout:172.21.15.68 smithi068.front.sepia.ceph.com smithi068 2019-11-25T09:03:27.511 INFO:teuthology.orchestra.run.smithi081:> sudo hostname smithi081 2019-11-25T09:03:27.662 INFO:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi068/smithi081/g' /etc/hosts 2019-11-25T09:03:27.720 INFO:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/172.21.15.68/172.21.15.81/g' /etc/hosts 2019-11-25T09:03:27.774 INFO:teuthology.orchestra.run.smithi081:> sudo sed -i -e 's/smithi068/smithi081/g' /etc/hostname 2019-11-25T09:03:27.826 INFO:teuthology.orchestra.run.smithi081:> sudo hostname smithi081 2019-11-25T09:03:27.880 INFO:teuthology.provision.fog.smithi081:Deploy complete! 2019-11-25T09:03:27.883 DEBUG:teuthology.parallel:result is None 2019-11-25T09:03:28.102 INFO:teuthology.lock.keys:Checking smithi153.front.sepia.ceph.com 2019-11-25T09:03:28.103 INFO:teuthology.lock.keys:New key found. Updating... 2019-11-25T09:03:28.154 INFO:teuthology.lock.keys:Checking smithi081.front.sepia.ceph.com 2019-11-25T09:03:28.154 INFO:teuthology.lock.keys:New key found. Updating... 2019-11-25T09:03:28.189 INFO:teuthology.lock.ops:Updating [smithi153.front.sepia.ceph.com]: set os type and version on server 2019-11-25T09:03:28.190 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-25T09:03:28.190 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:03:28.343 INFO:teuthology.orchestra.run.smithi153:> true 2019-11-25T09:03:28.726 INFO:teuthology.orchestra.run.smithi153:> uname -m 2019-11-25T09:03:28.805 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2019-11-25T09:03:28.806 INFO:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2019-11-25T09:03:28.907 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Linux" 2019-11-25T09:03:28.908 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="7 (Core)" 2019-11-25T09:03:28.908 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2019-11-25T09:03:28.908 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2019-11-25T09:03:28.908 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="7" 2019-11-25T09:03:28.908 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-25T09:03:28.908 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2019-11-25T09:03:28.909 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-25T09:03:28.909 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.centos.org/" 2019-11-25T09:03:28.909 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-25T09:03:28.909 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:28.909 INFO:teuthology.orchestra.run.smithi153.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-25T09:03:28.909 INFO:teuthology.orchestra.run.smithi153.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-25T09:03:28.909 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-25T09:03:28.910 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-25T09:03:28.910 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:28.911 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2019-11-25T09:03:28.924 INFO:teuthology.lock.ops:Updating [smithi081.front.sepia.ceph.com]: set os type and version on server 2019-11-25T09:03:28.924 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-25T09:03:28.924 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:03:29.046 INFO:teuthology.orchestra.run.smithi081:> true 2019-11-25T09:03:29.315 INFO:teuthology.orchestra.run.smithi081:> uname -m 2019-11-25T09:03:29.357 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2019-11-25T09:03:29.358 INFO:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2019-11-25T09:03:29.398 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Linux" 2019-11-25T09:03:29.399 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="7 (Core)" 2019-11-25T09:03:29.399 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2019-11-25T09:03:29.399 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2019-11-25T09:03:29.399 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="7" 2019-11-25T09:03:29.399 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-25T09:03:29.399 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2019-11-25T09:03:29.399 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-25T09:03:29.400 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.centos.org/" 2019-11-25T09:03:29.400 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-25T09:03:29.400 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:03:29.400 INFO:teuthology.orchestra.run.smithi081.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-25T09:03:29.400 INFO:teuthology.orchestra.run.smithi081.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-25T09:03:29.400 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-25T09:03:29.400 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-25T09:03:29.401 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:03:29.401 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2019-11-25T09:03:29.416 INFO:teuthology.task.internal.lock_machines:2 smithi machines locked this try, 2/2 locked so far 2019-11-25T09:03:29.442 INFO:teuthology.task.internal.lock_machines:Locked targets: smithi081.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDulLtKBKKywL9l8dfPo+H+fQDMdFzYGQ7+KrMuzE1hl2i63HEGzGR2FHjQHLDiwJ4CQvJTKBiOfDaRzQaLWaPpT5S6hzxvUvZE6EvOSwFS5Ql3OWrx3XwJCn91GW6I8PxO7Tjj9CORQFjIym5HyKwrsP97/1ZohFL+BRHZMuSxTazM07PHdTf+NtZPqtaeWoydHb8z6R143o0VcpLe9Nycbv5fWyE5GgSBomd9G4n+uHDCJ9owpWZKx8WmvtSb7D4cZW+I5mSPCmzTIwh7cvXYl9rF72PAcentXWHRWJ5nBWe5IzI3yNg5yzg8yIKfH8BCoO2pykVeDErsBeb/qEkf smithi153.front.sepia.ceph.com: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbJ3CA33GkUFO5ip99XQlSODRE6lM/dGjTcdtOM3LFHo8ORrhEGq6shai2YwAzGWaPyBaIKHEBHrpou7Rnkd3wHMtJRL2I0V4MOV/xOKCqWhSkowB2zlt53QSjR+2Pn0o558mhE/aWKcqAsd9OLU8mULVMutLYY3krVOK4ompYHGkJc+KF3x8KvYJ1H4PCp2oWYYbXWmqAb7+636SaD/fgmDSICeUhsHcWYpIsLdGDAiV47zyCvcO/Kyn00KdUpDDZkII36EKEO4fLCZ9yeQ3f8Hcn1DmJtnFTMIJAWm4xDMtjz+T/OEkXYyJkWTqXdxEATcgmps2VcwWfvgGxZ4AP 2019-11-25T09:03:29.443 DEBUG:teuthology.report:Pushing job info to http://paddles.front.sepia.ceph.com/ 2019-11-25T09:03:29.498 INFO:teuthology.run_tasks:Running task internal.save_config... 2019-11-25T09:03:29.516 INFO:teuthology.task.internal:Saving configuration 2019-11-25T09:03:29.534 INFO:teuthology.run_tasks:Running task internal.check_lock... 2019-11-25T09:03:29.555 INFO:teuthology.task.internal.check_lock:Checking locks... 2019-11-25T09:03:29.570 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-11-25 08:58:28.671700', u'locked_by': u'scheduled_yuvalif@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi153.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOzjCYhbtR68OsMcAsndjDMuXwhNUJvtGBrDyfkbOsOi5X2c5qdHs1tubYr4QCGADe/Bp2U3lAHY25oxT7i2/nw=', u'description': u'/home/teuthworker/archive/yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi/4542041'} 2019-11-25T09:03:29.583 DEBUG:teuthology.task.internal.check_lock:machine status is {u'is_vm': False, u'locked': True, u'locked_since': u'2019-11-25 08:58:28.671681', u'locked_by': u'scheduled_yuvalif@teuthology', u'up': True, u'mac_address': None, u'name': u'smithi081.front.sepia.ceph.com', u'os_version': u'7', u'machine_type': u'smithi', u'vm_host': None, u'os_type': u'centos', u'arch': u'x86_64', u'ssh_pub_key': u'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNMzPOJeRb8pgCV/rj66zfQz3NWL+0h2UXgwgUGi13lQ6CWQ/g6JvjkojzE29B+ackdh8QUU5Jo1hi45wQQ79vc=', u'description': u'/home/teuthworker/archive/yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi/4542041'} 2019-11-25T09:03:29.583 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2019-11-25T09:03:29.590 INFO:teuthology.task.internal:roles: ubuntu@smithi153.front.sepia.ceph.com - ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-11-25T09:03:29.590 INFO:teuthology.task.internal:roles: ubuntu@smithi081.front.sepia.ceph.com - ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-11-25T09:03:29.590 INFO:teuthology.run_tasks:Running task console_log... 2019-11-25T09:03:29.756 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=, signals=[15]) 2019-11-25T09:03:29.757 INFO:teuthology.run_tasks:Running task internal.connect... 2019-11-25T09:03:29.769 INFO:teuthology.task.internal:Opening connections... 2019-11-25T09:03:29.769 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:03:29.772 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:03:29.870 DEBUG:teuthology.task.internal:connecting to ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:03:29.871 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:03:29.979 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2019-11-25T09:03:29.987 INFO:teuthology.orchestra.run.smithi153:> uname -m 2019-11-25T09:03:30.171 INFO:teuthology.orchestra.run.smithi153.stdout:x86_64 2019-11-25T09:03:30.171 INFO:teuthology.orchestra.run.smithi153:> cat /etc/os-release 2019-11-25T09:03:30.222 INFO:teuthology.orchestra.run.smithi153.stdout:NAME="CentOS Linux" 2019-11-25T09:03:30.222 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION="7 (Core)" 2019-11-25T09:03:30.222 INFO:teuthology.orchestra.run.smithi153.stdout:ID="centos" 2019-11-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi153.stdout:ID_LIKE="rhel fedora" 2019-11-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi153.stdout:VERSION_ID="7" 2019-11-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi153.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi153.stdout:ANSI_COLOR="0;31" 2019-11-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi153.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-25T09:03:30.223 INFO:teuthology.orchestra.run.smithi153.stdout:HOME_URL="https://www.centos.org/" 2019-11-25T09:03:30.224 INFO:teuthology.orchestra.run.smithi153.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-25T09:03:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:30.224 INFO:teuthology.orchestra.run.smithi153.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-25T09:03:30.224 INFO:teuthology.orchestra.run.smithi153.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-25T09:03:30.224 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-25T09:03:30.224 INFO:teuthology.orchestra.run.smithi153.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-25T09:03:30.224 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:30.225 INFO:teuthology.lock.ops:Updating smithi153.front.sepia.ceph.com on lock server 2019-11-25T09:03:30.240 INFO:teuthology.orchestra.run.smithi081:> uname -m 2019-11-25T09:03:30.304 INFO:teuthology.orchestra.run.smithi081.stdout:x86_64 2019-11-25T09:03:30.304 INFO:teuthology.orchestra.run.smithi081:> cat /etc/os-release 2019-11-25T09:03:30.346 INFO:teuthology.orchestra.run.smithi081.stdout:NAME="CentOS Linux" 2019-11-25T09:03:30.347 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION="7 (Core)" 2019-11-25T09:03:30.347 INFO:teuthology.orchestra.run.smithi081.stdout:ID="centos" 2019-11-25T09:03:30.347 INFO:teuthology.orchestra.run.smithi081.stdout:ID_LIKE="rhel fedora" 2019-11-25T09:03:30.347 INFO:teuthology.orchestra.run.smithi081.stdout:VERSION_ID="7" 2019-11-25T09:03:30.347 INFO:teuthology.orchestra.run.smithi081.stdout:PRETTY_NAME="CentOS Linux 7 (Core)" 2019-11-25T09:03:30.348 INFO:teuthology.orchestra.run.smithi081.stdout:ANSI_COLOR="0;31" 2019-11-25T09:03:30.348 INFO:teuthology.orchestra.run.smithi081.stdout:CPE_NAME="cpe:/o:centos:centos:7" 2019-11-25T09:03:30.348 INFO:teuthology.orchestra.run.smithi081.stdout:HOME_URL="https://www.centos.org/" 2019-11-25T09:03:30.348 INFO:teuthology.orchestra.run.smithi081.stdout:BUG_REPORT_URL="https://bugs.centos.org/" 2019-11-25T09:03:30.348 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:03:30.348 INFO:teuthology.orchestra.run.smithi081.stdout:CENTOS_MANTISBT_PROJECT="CentOS-7" 2019-11-25T09:03:30.348 INFO:teuthology.orchestra.run.smithi081.stdout:CENTOS_MANTISBT_PROJECT_VERSION="7" 2019-11-25T09:03:30.348 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT="centos" 2019-11-25T09:03:30.349 INFO:teuthology.orchestra.run.smithi081.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="7" 2019-11-25T09:03:30.349 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:03:30.349 INFO:teuthology.lock.ops:Updating smithi081.front.sepia.ceph.com on lock server 2019-11-25T09:03:30.362 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2019-11-25T09:03:30.379 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2019-11-25T09:03:30.391 INFO:teuthology.task.internal:Checking for old test directory... 2019-11-25T09:03:30.391 INFO:teuthology.orchestra.run.smithi081:> test '!' -e /home/ubuntu/cephtest 2019-11-25T09:03:30.396 INFO:teuthology.orchestra.run.smithi153:> test '!' -e /home/ubuntu/cephtest 2019-11-25T09:03:30.711 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2019-11-25T09:03:30.726 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2019-11-25T09:03:30.727 INFO:teuthology.orchestra.run.smithi081:> test -z $(ls -A /var/lib/ceph) 2019-11-25T09:03:30.734 INFO:teuthology.orchestra.run.smithi153:> test -z $(ls -A /var/lib/ceph) 2019-11-25T09:03:30.828 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2019-11-25T09:03:30.890 INFO:teuthology.run_tasks:Running task kernel... 2019-11-25T09:03:30.930 DEBUG:teuthology.task.kernel:normalized config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.c': {'sha1': 'distro', 'kdb': True}, 'client.0': {'sha1': 'distro', 'kdb': True}, 'osd.1': {'sha1': 'distro', 'kdb': True}, 'osd.0': {'sha1': 'distro', 'kdb': True}, 'osd.3': {'sha1': 'distro', 'kdb': True}, 'osd.2': {'sha1': 'distro', 'kdb': True}, 'osd.5': {'sha1': 'distro', 'kdb': True}, 'osd.4': {'sha1': 'distro', 'kdb': True}, 'osd.7': {'sha1': 'distro', 'kdb': True}, 'osd.6': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}, 'mgr.y': {'sha1': 'distro', 'kdb': True}, 'mgr.x': {'sha1': 'distro', 'kdb': True}, 'client.1': {'sha1': 'distro', 'kdb': True}} 2019-11-25T09:03:30.931 INFO:teuthology.task.kernel:config {'mon.a': {'sha1': 'distro', 'kdb': True}, 'mon.b': {'sha1': 'distro', 'kdb': True}}, timeout 300 2019-11-25T09:03:30.945 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:03:30.945 INFO:teuthology.orchestra.run.smithi153:> sudo package-cleanup -y --oldkernels 2019-11-25T09:03:31.920 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:03:32.907 INFO:teuthology.orchestra.run.smithi153.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:32.908 INFO:teuthology.orchestra.run.smithi153.stdout:Trying other mirror. 2019-11-25T09:03:32.908 INFO:teuthology.orchestra.run.smithi153.stdout:To address this issue please refer to the below wiki article 2019-11-25T09:03:32.908 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:32.908 INFO:teuthology.orchestra.run.smithi153.stdout:https://wiki.centos.org/yum-errors 2019-11-25T09:03:32.908 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:32.909 INFO:teuthology.orchestra.run.smithi153.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-11-25T09:03:32.909 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:33.078 INFO:teuthology.orchestra.run.smithi153.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:33.079 INFO:teuthology.orchestra.run.smithi153.stdout:Trying other mirror. 2019-11-25T09:03:33.172 INFO:teuthology.orchestra.run.smithi153.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:33.172 INFO:teuthology.orchestra.run.smithi153.stdout:Trying other mirror. 2019-11-25T09:03:33.435 INFO:teuthology.orchestra.run.smithi153.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:33.436 INFO:teuthology.orchestra.run.smithi153.stdout:Trying other mirror. 2019-11-25T09:03:33.595 INFO:teuthology.orchestra.run.smithi153.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:33.595 INFO:teuthology.orchestra.run.smithi153.stdout:Trying other mirror. 2019-11-25T09:03:35.047 INFO:teuthology.orchestra.run.smithi153.stdout:No old kernels to remove 2019-11-25T09:03:35.068 INFO:teuthology.task.kernel:Removing old kernels from ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:03:35.068 INFO:teuthology.orchestra.run.smithi081:> sudo package-cleanup -y --oldkernels 2019-11-25T09:03:35.997 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:03:37.202 INFO:teuthology.orchestra.run.smithi081.stdout:http://ftp.linux.ncsu.edu/pub/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:37.202 INFO:teuthology.orchestra.run.smithi081.stdout:Trying other mirror. 2019-11-25T09:03:37.202 INFO:teuthology.orchestra.run.smithi081.stdout:To address this issue please refer to the below wiki article 2019-11-25T09:03:37.202 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:03:37.203 INFO:teuthology.orchestra.run.smithi081.stdout:https://wiki.centos.org/yum-errors 2019-11-25T09:03:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:03:37.203 INFO:teuthology.orchestra.run.smithi081.stdout:If above article doesn't help to resolve this issue please use https://bugs.centos.org/. 2019-11-25T09:03:37.203 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:03:37.348 INFO:teuthology.orchestra.run.smithi081.stdout:http://dl.fedoraproject.org/pub/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:37.349 INFO:teuthology.orchestra.run.smithi081.stdout:Trying other mirror. 2019-11-25T09:03:37.441 INFO:teuthology.orchestra.run.smithi081.stdout:http://mirror.oss.ou.edu/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:37.442 INFO:teuthology.orchestra.run.smithi081.stdout:Trying other mirror. 2019-11-25T09:03:37.585 INFO:teuthology.orchestra.run.smithi081.stdout:http://mirrors.cat.pdx.edu/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:37.585 INFO:teuthology.orchestra.run.smithi081.stdout:Trying other mirror. 2019-11-25T09:03:37.737 INFO:teuthology.orchestra.run.smithi081.stdout:http://mirror.pnl.gov/epel/7/x86_64/repodata/733703470dd10accd9234492acb0c74de09f91772c2767107f6b0fa567cbc26a-updateinfo.xml.bz2: [Errno 14] HTTP Error 404 - Not Found 2019-11-25T09:03:37.738 INFO:teuthology.orchestra.run.smithi081.stdout:Trying other mirror. 2019-11-25T09:03:39.200 INFO:teuthology.orchestra.run.smithi081.stdout:No old kernels to remove 2019-11-25T09:03:39.221 INFO:teuthology.orchestra.run.smithi153:> uname -r 2019-11-25T09:03:39.336 INFO:teuthology.orchestra.run.smithi153.stdout:3.10.0-1062.4.1.el7.x86_64 2019-11-25T09:03:39.337 INFO:teuthology.task.kernel:Running kernel on smithi153: 3.10.0-1062.4.1.el7.x86_64 2019-11-25T09:03:39.337 INFO:teuthology.orchestra.run.smithi153:> sudo yum install -y kernel 2019-11-25T09:03:39.587 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:03:39.627 INFO:teuthology.orchestra.run.smithi153.stdout:Determining fastest mirrors 2019-11-25T09:03:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:03:39.788 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:03:39.789 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:03:39.790 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:03:41.136 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:03:41.136 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:03:41.136 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package kernel.x86_64 0:3.10.0-1062.4.3.el7 will be installed 2019-11-25T09:03:42.468 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:03:43.342 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:43.342 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:03:43.364 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:43.364 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:03:43.364 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T09:03:43.364 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:03:43.364 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:03:43.364 INFO:teuthology.orchestra.run.smithi153.stdout: kernel x86_64 3.10.0-1062.4.3.el7 updates 50 M 2019-11-25T09:03:43.365 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:43.365 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:03:43.365 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:03:43.365 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2019-11-25T09:03:43.365 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:03:43.365 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 50 M 2019-11-25T09:03:43.365 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 64 M 2019-11-25T09:03:43.366 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:03:43.371 INFO:teuthology.orchestra.run.smithi153.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-25T09:03:44.687 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:03:44.882 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:03:45.837 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:03:45.838 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:04:21.455 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : kernel-3.10.0-1062.4.3.el7.x86_64 1/1 2019-11-25T09:04:21.944 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : kernel-3.10.0-1062.4.3.el7.x86_64 1/1 2019-11-25T09:04:21.945 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:04:21.945 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:04:21.945 INFO:teuthology.orchestra.run.smithi153.stdout: kernel.x86_64 0:3.10.0-1062.4.3.el7 2019-11-25T09:04:21.945 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:04:21.945 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:04:21.995 INFO:teuthology.orchestra.run.smithi153:> rpm -q kernel --last | head -n 1 2019-11-25T09:04:22.397 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Mon 25 Nov 2019 09:03:48 AM UTC 2019-11-25T09:04:22.400 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:04:22.400 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:04:22.400 INFO:teuthology.orchestra.run.smithi081:> uname -r 2019-11-25T09:04:22.520 INFO:teuthology.orchestra.run.smithi081.stdout:3.10.0-1062.4.1.el7.x86_64 2019-11-25T09:04:22.521 INFO:teuthology.task.kernel:Running kernel on smithi081: 3.10.0-1062.4.1.el7.x86_64 2019-11-25T09:04:22.521 INFO:teuthology.orchestra.run.smithi081:> sudo yum install -y kernel 2019-11-25T09:04:22.776 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:04:22.819 INFO:teuthology.orchestra.run.smithi081.stdout:Determining fastest mirrors 2019-11-25T09:04:22.976 INFO:teuthology.orchestra.run.smithi081.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:04:22.977 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:04:22.978 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:04:22.978 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.trouble-free.net 2019-11-25T09:04:24.477 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:04:24.478 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:04:24.478 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package kernel.x86_64 0:3.10.0-1062.4.3.el7 will be installed 2019-11-25T09:04:25.905 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:04:26.815 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:04:26.815 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:04:26.827 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:04:26.827 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:04:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2019-11-25T09:04:26.828 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:04:26.828 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:04:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: kernel x86_64 3.10.0-1062.4.3.el7 updates 50 M 2019-11-25T09:04:26.828 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:04:26.828 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:04:26.829 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:04:26.829 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2019-11-25T09:04:26.829 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:04:26.829 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 50 M 2019-11-25T09:04:26.829 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 64 M 2019-11-25T09:04:26.829 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:04:26.838 INFO:teuthology.orchestra.run.smithi081.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-25T09:04:31.756 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:04:31.950 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:04:32.898 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:04:32.898 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:05:08.151 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : kernel-3.10.0-1062.4.3.el7.x86_64 1/1 2019-11-25T09:05:08.685 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : kernel-3.10.0-1062.4.3.el7.x86_64 1/1 2019-11-25T09:05:08.685 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:05:08.685 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:05:08.685 INFO:teuthology.orchestra.run.smithi081.stdout: kernel.x86_64 0:3.10.0-1062.4.3.el7 2019-11-25T09:05:08.685 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:05:08.685 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:05:08.733 INFO:teuthology.orchestra.run.smithi081:> rpm -q kernel --last | head -n 1 2019-11-25T09:05:09.111 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Mon 25 Nov 2019 09:04:35 AM UTC 2019-11-25T09:05:09.112 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:05:09.112 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 3.10.0-1062.4.1.el7.x86_64 Expected: 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:05:09.112 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2019-11-25T09:05:09.112 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-11-25T09:05:09.112 DEBUG:teuthology.task.kernel:src is distro, skipping download 2019-11-25T09:05:09.112 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2019-11-25T09:05:09.112 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi153.front.sepia.ceph.com, path=None, version=distro) 2019-11-25T09:05:09.113 INFO:teuthology.orchestra.run.smithi153:> rpm -q kernel --last | head -n 1 2019-11-25T09:05:09.489 INFO:teuthology.orchestra.run.smithi153.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Mon 25 Nov 2019 09:03:48 AM UTC 2019-11-25T09:05:09.491 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:05:09.491 INFO:teuthology.orchestra.run.smithi153:> sudo rpm -qi grub2 2019-11-25T09:05:09.654 INFO:teuthology.orchestra.run.smithi153.stdout:Name : grub2 2019-11-25T09:05:09.655 INFO:teuthology.orchestra.run.smithi153.stdout:Epoch : 1 2019-11-25T09:05:09.655 INFO:teuthology.orchestra.run.smithi153.stdout:Version : 2.02 2019-11-25T09:05:09.655 INFO:teuthology.orchestra.run.smithi153.stdout:Release : 0.76.el7.centos 2019-11-25T09:05:09.656 INFO:teuthology.orchestra.run.smithi153.stdout:Architecture: x86_64 2019-11-25T09:05:09.656 INFO:teuthology.orchestra.run.smithi153.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-11-25T09:05:09.656 INFO:teuthology.orchestra.run.smithi153.stdout:Group : System Environment/Base 2019-11-25T09:05:09.656 INFO:teuthology.orchestra.run.smithi153.stdout:Size : 0 2019-11-25T09:05:09.657 INFO:teuthology.orchestra.run.smithi153.stdout:License : GPLv3+ 2019-11-25T09:05:09.657 INFO:teuthology.orchestra.run.smithi153.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-11-25T09:05:09.657 INFO:teuthology.orchestra.run.smithi153.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-11-25T09:05:09.658 INFO:teuthology.orchestra.run.smithi153.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-11-25T09:05:09.658 INFO:teuthology.orchestra.run.smithi153.stdout:Build Host : kbuilder.bsys.centos.org 2019-11-25T09:05:09.658 INFO:teuthology.orchestra.run.smithi153.stdout:Relocations : (not relocatable) 2019-11-25T09:05:09.658 INFO:teuthology.orchestra.run.smithi153.stdout:Packager : CentOS BuildSystem 2019-11-25T09:05:09.658 INFO:teuthology.orchestra.run.smithi153.stdout:Vendor : CentOS 2019-11-25T09:05:09.659 INFO:teuthology.orchestra.run.smithi153.stdout:URL : http://www.gnu.org/software/grub/ 2019-11-25T09:05:09.659 INFO:teuthology.orchestra.run.smithi153.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-11-25T09:05:09.659 INFO:teuthology.orchestra.run.smithi153.stdout:Description : 2019-11-25T09:05:09.659 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:05:09.660 INFO:teuthology.orchestra.run.smithi153.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-11-25T09:05:09.660 INFO:teuthology.orchestra.run.smithi153.stdout:customizable bootloader with modular architecture. It supports a rich 2019-11-25T09:05:09.660 INFO:teuthology.orchestra.run.smithi153.stdout:variety of kernel formats, file systems, computer architectures and 2019-11-25T09:05:09.660 INFO:teuthology.orchestra.run.smithi153.stdout:hardware devices. 2019-11-25T09:05:09.662 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-11-25T09:05:09.662 INFO:teuthology.task.kernel:Updating grub on smithi153 to boot 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:05:09.662 INFO:teuthology.orchestra.run.smithi153:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-11-25T09:05:10.153 INFO:teuthology.orchestra.run.smithi153.stderr:Generating grub configuration file ... 2019-11-25T09:05:10.310 INFO:teuthology.orchestra.run.smithi153.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:05:10.326 INFO:teuthology.orchestra.run.smithi153.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.3.el7.x86_64.img 2019-11-25T09:05:10.602 INFO:teuthology.orchestra.run.smithi153.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-11-25T09:05:10.628 INFO:teuthology.orchestra.run.smithi153.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-11-25T09:05:10.664 INFO:teuthology.orchestra.run.smithi153.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-11-25T09:05:10.688 INFO:teuthology.orchestra.run.smithi153.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-11-25T09:05:10.716 INFO:teuthology.orchestra.run.smithi153.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-11-25T09:05:10.726 INFO:teuthology.orchestra.run.smithi153.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-11-25T09:05:11.862 INFO:teuthology.orchestra.run.smithi153.stderr:done 2019-11-25T09:05:11.864 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:05:12.013 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.0XcI1O4lyv 2019-11-25T09:05:12.014 INFO:teuthology.orchestra.run.smithi153:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.0XcI1O4lyv 2019-11-25T09:05:12.126 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.0XcI1O4lyv 2019-11-25T09:05:12.365 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.0XcI1O4lyv is 6KB 2019-11-25T09:05:12.446 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.0XcI1O4lyv 2019-11-25T09:05:12.515 INFO:teuthology.orchestra.run.smithi153:> sudo grub2-set-default 0 2019-11-25T09:05:12.674 INFO:teuthology.orchestra.run.smithi153:> sudo shutdown -r now 2019-11-25T09:05:12.687 INFO:teuthology.task.kernel:Installing distro kernel on mon.b... 2019-11-25T09:05:12.687 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@smithi081.front.sepia.ceph.com, path=None, version=distro) 2019-11-25T09:05:12.687 INFO:teuthology.orchestra.run.smithi081:> rpm -q kernel --last | head -n 1 2019-11-25T09:05:13.055 INFO:teuthology.orchestra.run.smithi081.stdout:kernel-3.10.0-1062.4.3.el7.x86_64 Mon 25 Nov 2019 09:04:35 AM UTC 2019-11-25T09:05:13.056 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:05:13.057 INFO:teuthology.orchestra.run.smithi081:> sudo rpm -qi grub2 2019-11-25T09:05:13.202 INFO:teuthology.orchestra.run.smithi081.stdout:Name : grub2 2019-11-25T09:05:13.202 INFO:teuthology.orchestra.run.smithi081.stdout:Epoch : 1 2019-11-25T09:05:13.202 INFO:teuthology.orchestra.run.smithi081.stdout:Version : 2.02 2019-11-25T09:05:13.203 INFO:teuthology.orchestra.run.smithi081.stdout:Release : 0.76.el7.centos 2019-11-25T09:05:13.203 INFO:teuthology.orchestra.run.smithi081.stdout:Architecture: x86_64 2019-11-25T09:05:13.203 INFO:teuthology.orchestra.run.smithi081.stdout:Install Date: Tue 05 Nov 2019 03:41:14 PM UTC 2019-11-25T09:05:13.203 INFO:teuthology.orchestra.run.smithi081.stdout:Group : System Environment/Base 2019-11-25T09:05:13.203 INFO:teuthology.orchestra.run.smithi081.stdout:Size : 0 2019-11-25T09:05:13.203 INFO:teuthology.orchestra.run.smithi081.stdout:License : GPLv3+ 2019-11-25T09:05:13.206 INFO:teuthology.orchestra.run.smithi081.stdout:Signature : RSA/SHA256, Mon 12 Nov 2018 02:28:06 PM UTC, Key ID 24c6a8a7f4a80eb5 2019-11-25T09:05:13.206 INFO:teuthology.orchestra.run.smithi081.stdout:Source RPM : grub2-2.02-0.76.el7.centos.src.rpm 2019-11-25T09:05:13.206 INFO:teuthology.orchestra.run.smithi081.stdout:Build Date : Fri 09 Nov 2018 12:58:43 AM UTC 2019-11-25T09:05:13.206 INFO:teuthology.orchestra.run.smithi081.stdout:Build Host : kbuilder.bsys.centos.org 2019-11-25T09:05:13.206 INFO:teuthology.orchestra.run.smithi081.stdout:Relocations : (not relocatable) 2019-11-25T09:05:13.207 INFO:teuthology.orchestra.run.smithi081.stdout:Packager : CentOS BuildSystem 2019-11-25T09:05:13.207 INFO:teuthology.orchestra.run.smithi081.stdout:Vendor : CentOS 2019-11-25T09:05:13.207 INFO:teuthology.orchestra.run.smithi081.stdout:URL : http://www.gnu.org/software/grub/ 2019-11-25T09:05:13.207 INFO:teuthology.orchestra.run.smithi081.stdout:Summary : Bootloader with support for Linux, Multiboot and more 2019-11-25T09:05:13.207 INFO:teuthology.orchestra.run.smithi081.stdout:Description : 2019-11-25T09:05:13.207 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:05:13.207 INFO:teuthology.orchestra.run.smithi081.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2019-11-25T09:05:13.208 INFO:teuthology.orchestra.run.smithi081.stdout:customizable bootloader with modular architecture. It supports a rich 2019-11-25T09:05:13.208 INFO:teuthology.orchestra.run.smithi081.stdout:variety of kernel formats, file systems, computer architectures and 2019-11-25T09:05:13.208 INFO:teuthology.orchestra.run.smithi081.stdout:hardware devices. 2019-11-25T09:05:13.209 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2019-11-25T09:05:13.209 INFO:teuthology.task.kernel:Updating grub on smithi081 to boot 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:05:13.209 INFO:teuthology.orchestra.run.smithi081:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2019-11-25T09:05:13.682 INFO:teuthology.orchestra.run.smithi081.stderr:Generating grub configuration file ... 2019-11-25T09:05:13.859 INFO:teuthology.orchestra.run.smithi081.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:05:13.875 INFO:teuthology.orchestra.run.smithi081.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.3.el7.x86_64.img 2019-11-25T09:05:14.162 INFO:teuthology.orchestra.run.smithi081.stderr:Found linux image: /boot/vmlinuz-3.10.0-1062.4.1.el7.x86_64 2019-11-25T09:05:14.205 INFO:teuthology.orchestra.run.smithi081.stderr:Found initrd image: /boot/initramfs-3.10.0-1062.4.1.el7.x86_64.img 2019-11-25T09:05:14.247 INFO:teuthology.orchestra.run.smithi081.stderr:Found linux image: /boot/vmlinuz-3.10.0-957.el7.x86_64 2019-11-25T09:05:14.268 INFO:teuthology.orchestra.run.smithi081.stderr:Found initrd image: /boot/initramfs-3.10.0-957.el7.x86_64.img 2019-11-25T09:05:14.303 INFO:teuthology.orchestra.run.smithi081.stderr:Found linux image: /boot/vmlinuz-0-rescue-a1739c9c60b14f0891126d2e603bb003 2019-11-25T09:05:14.315 INFO:teuthology.orchestra.run.smithi081.stderr:Found initrd image: /boot/initramfs-0-rescue-a1739c9c60b14f0891126d2e603bb003.img 2019-11-25T09:05:15.523 INFO:teuthology.orchestra.run.smithi081.stderr:done 2019-11-25T09:05:15.527 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:05:15.590 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.4LH8GIjFW7 2019-11-25T09:05:15.591 INFO:teuthology.orchestra.run.smithi081:> sudo cp /boot/grub2/grub.cfg /tmp/tmp.4LH8GIjFW7 2019-11-25T09:05:15.661 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.4LH8GIjFW7 2019-11-25T09:05:15.779 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.4LH8GIjFW7 is 6KB 2019-11-25T09:05:15.831 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.4LH8GIjFW7 2019-11-25T09:05:15.877 INFO:teuthology.orchestra.run.smithi081:> sudo grub2-set-default 0 2019-11-25T09:05:16.126 INFO:teuthology.orchestra.run.smithi081:> sudo shutdown -r now 2019-11-25T09:05:16.131 INFO:teuthology.misc:Re-opening connections... 2019-11-25T09:05:16.131 INFO:teuthology.misc:trying to connect to ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:05:16.131 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-25T09:05:16.132 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:06:16.133 DEBUG:teuthology.orchestra.remote:timed out 2019-11-25T09:06:16.134 INFO:teuthology.misc:trying to connect to ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:06:16.134 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-25T09:06:16.134 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:07:16.135 DEBUG:teuthology.orchestra.remote:timed out 2019-11-25T09:07:16.136 DEBUG:teuthology.misc:waited 120.004320145 2019-11-25T09:07:17.136 INFO:teuthology.misc:trying to connect to ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:07:17.136 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-25T09:07:17.137 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:07:17.236 INFO:teuthology.orchestra.run.smithi153:> true 2019-11-25T09:07:17.710 DEBUG:teuthology.misc:waited 121.578598022 2019-11-25T09:07:18.710 INFO:teuthology.misc:trying to connect to ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:07:18.711 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-25T09:07:18.711 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:07:18.814 INFO:teuthology.orchestra.run.smithi081:> true 2019-11-25T09:07:19.123 DEBUG:teuthology.misc:waited 122.99172616 2019-11-25T09:07:20.123 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2019-11-25T09:07:20.124 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "3.10.0-1062.4.3.el7.x86_64"... 2019-11-25T09:07:20.124 INFO:teuthology.orchestra.run.smithi153:> uname -r 2019-11-25T09:07:20.229 INFO:teuthology.orchestra.run.smithi153.stdout:3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:07:20.229 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:07:20.229 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-11-25T09:07:20.230 INFO:teuthology.task.kernel:Checking client mon.b for new kernel version... 2019-11-25T09:07:20.230 INFO:teuthology.task.kernel:Checking kernel version of mon.b, want "3.10.0-1062.4.3.el7.x86_64"... 2019-11-25T09:07:20.230 INFO:teuthology.orchestra.run.smithi081:> uname -r 2019-11-25T09:07:20.311 INFO:teuthology.orchestra.run.smithi081.stdout:3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:07:20.312 DEBUG:teuthology.task.kernel:current kernel version is 3.10.0-1062.4.3.el7.x86_64 2019-11-25T09:07:20.312 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2019-11-25T09:07:21.323 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2019-11-25T09:07:21.323 INFO:teuthology.orchestra.run.smithi153:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-11-25T09:07:21.503 INFO:teuthology.orchestra.run.smithi153.stdout:ttyS1 2019-11-25T09:07:21.515 INFO:teuthology.task.kernel:Enabling kdb on mon.b... 2019-11-25T09:07:21.515 INFO:teuthology.orchestra.run.smithi081:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2019-11-25T09:07:21.613 INFO:teuthology.orchestra.run.smithi081.stdout:ttyS1 2019-11-25T09:07:21.623 INFO:teuthology.run_tasks:Running task internal.base... 2019-11-25T09:07:21.675 INFO:teuthology.task.internal:Creating test directory... 2019-11-25T09:07:21.675 INFO:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-11-25T09:07:21.682 INFO:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2019-11-25T09:07:21.776 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2019-11-25T09:07:21.865 INFO:teuthology.run_tasks:Running task internal.archive... 2019-11-25T09:07:21.885 INFO:teuthology.task.internal:Creating archive directory... 2019-11-25T09:07:21.886 INFO:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-11-25T09:07:21.893 INFO:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2019-11-25T09:07:21.999 INFO:teuthology.run_tasks:Running task internal.coredump... 2019-11-25T09:07:22.021 INFO:teuthology.task.internal:Enabling coredump saving... 2019-11-25T09:07:22.021 INFO:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-11-25T09:07:22.026 INFO:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-11-25T09:07:22.078 INFO:teuthology.orchestra.run.smithi081.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-11-25T09:07:22.147 INFO:teuthology.orchestra.run.smithi153.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2019-11-25T09:07:22.150 INFO:teuthology.run_tasks:Running task internal.sudo... 2019-11-25T09:07:22.161 INFO:teuthology.task.internal:Configuring sudo... 2019-11-25T09:07:22.162 INFO:teuthology.orchestra.run.smithi081:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-11-25T09:07:22.166 INFO:teuthology.orchestra.run.smithi153:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2019-11-25T09:07:22.260 INFO:teuthology.run_tasks:Running task internal.syslog... 2019-11-25T09:07:22.272 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2019-11-25T09:07:22.272 INFO:teuthology.orchestra.run.smithi081:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-11-25T09:07:22.276 INFO:teuthology.orchestra.run.smithi153:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2019-11-25T09:07:22.343 INFO:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-11-25T09:07:22.421 INFO:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-11-25T09:07:22.508 INFO:teuthology.orchestra.run.smithi153:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-11-25T09:07:22.577 INFO:teuthology.orchestra.run.smithi153:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-11-25T09:07:22.644 INFO:teuthology.orchestra.run.smithi153:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-11-25T09:07:22.708 INFO:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2019-11-25T09:07:22.766 INFO:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2019-11-25T09:07:22.925 INFO:teuthology.orchestra.run.smithi081:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2019-11-25T09:07:22.997 INFO:teuthology.orchestra.run.smithi081:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2019-11-25T09:07:23.061 INFO:teuthology.orchestra.run.smithi081:> sudo sh -c 'cat > /etc/rsyslog.d/80-cephtest.conf' 2019-11-25T09:07:23.122 INFO:teuthology.orchestra.run.smithi081:> sudo service rsyslog restart 2019-11-25T09:07:23.189 INFO:teuthology.orchestra.run.smithi153:> sudo service rsyslog restart 2019-11-25T09:07:23.290 INFO:teuthology.orchestra.run.smithi081.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-11-25T09:07:23.327 INFO:teuthology.orchestra.run.smithi153.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2019-11-25T09:07:23.385 INFO:teuthology.run_tasks:Running task internal.timer... 2019-11-25T09:07:23.399 INFO:teuthology.task.internal:Starting timer... 2019-11-25T09:07:23.401 INFO:teuthology.run_tasks:Running task pcp... 2019-11-25T09:07:23.432 INFO:teuthology.run_tasks:Running task selinux... 2019-11-25T09:07:23.445 INFO:teuthology.orchestra.run.smithi081:> sudo service auditd rotate 2019-11-25T09:07:23.635 INFO:teuthology.orchestra.run.smithi081.stdout:Rotating logs: [ OK ] 2019-11-25T09:07:23.639 INFO:teuthology.orchestra.run.smithi153:> sudo service auditd rotate 2019-11-25T09:07:23.806 INFO:teuthology.orchestra.run.smithi153.stdout:Rotating logs: [ OK ] 2019-11-25T09:07:23.809 DEBUG:teuthology.task.selinux:Getting current SELinux state 2019-11-25T09:07:23.809 INFO:teuthology.orchestra.run.smithi153:> /usr/sbin/getenforce 2019-11-25T09:07:23.899 INFO:teuthology.orchestra.run.smithi153.stdout:Permissive 2019-11-25T09:07:23.900 INFO:teuthology.orchestra.run.smithi081:> /usr/sbin/getenforce 2019-11-25T09:07:23.999 INFO:teuthology.orchestra.run.smithi081.stdout:Permissive 2019-11-25T09:07:23.999 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@smithi153.front.sepia.ceph.com': 'permissive', 'ubuntu@smithi081.front.sepia.ceph.com': 'permissive'} 2019-11-25T09:07:23.999 INFO:teuthology.orchestra.run.smithi153:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-11-25T09:07:24.118 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-11-25T09:07:24.118 INFO:teuthology.orchestra.run.smithi081:> sudo grep 'avc: .*denied' /var/log/audit/audit.log | grep -v '\(comm="dmidecode"\|chronyd.service\|name="cephtest"\|scontext=system_u:system_r:nrpe_t:s0\|scontext=system_u:system_r:pcp_pmlogger_t\|scontext=system_u:system_r:pcp_pmcd_t:s0\|comm="rhsmd"\|scontext=system_u:system_r:syslogd_t:s0\|tcontext=system_u:system_r:nrpe_t:s0\|comm="updatedb"\|comm="smartd"\)' 2019-11-25T09:07:24.241 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-11-25T09:07:24.241 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2019-11-25T09:07:24.241 INFO:teuthology.orchestra.run.smithi153:> sudo /usr/sbin/setenforce permissive 2019-11-25T09:07:24.379 INFO:teuthology.orchestra.run.smithi081:> sudo /usr/sbin/setenforce permissive 2019-11-25T09:07:24.505 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2019-11-25T09:07:24.560 INFO:teuthology.repo_utils:/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master was just updated; assuming it is current 2019-11-25T09:07:24.561 INFO:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master to branch origin/master 2019-11-25T09:07:24.602 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}] 2019-11-25T09:07:24.612 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit smithi153.front.sepia.ceph.com,smithi081.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/cephlab.yml 2019-11-25T09:07:24.996 INFO:teuthology.task.ansible.out:Using /home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/ansible.cfg as config file 2019-11-25T09:07:28.662 INFO:teuthology.task.ansible.out: PLAY [all] ********************************************************************* 2019-11-25T09:07:28.686 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:28 +0000 (0:00:00.066) 0:00:00.066 ******* 2019-11-25T09:07:28.698 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:28 +0000 (0:00:00.011) 0:00:00.078 ******* 2019-11-25T09:07:31.872 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi153.front.sepia.ceph.com] 2019-11-25T09:07:31.917 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] 2019-11-25T09:07:31.995 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:31 +0000 (0:00:03.294) 0:00:03.373 ******* 2019-11-25T09:07:32.040 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:32 +0000 (0:00:00.047) 0:00:03.421 ******* 2019-11-25T09:07:32.432 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the sudo group.] ******************************** 2019-11-25T09:07:32.433 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-11-25T09:07:32.470 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:32 +0000 (0:00:00.429) 0:00:03.850 ******* 2019-11-25T09:07:32.477 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-11-25T09:07:32.514 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:32 +0000 (0:00:00.044) 0:00:03.895 ******* 2019-11-25T09:07:33.049 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the ansible user.] ****************************** 2019-11-25T09:07:33.051 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-11-25T09:07:33.088 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:33 +0000 (0:00:00.574) 0:00:04.469 ******* 2019-11-25T09:07:33.095 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 1100, "groups": "sudo", "home": "/home/cm", "move_home": false, "name": "cm", "shell": "/bin/bash", "state": "present", "uid": 1100} 2019-11-25T09:07:33.142 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:33 +0000 (0:00:00.053) 0:00:04.523 ******* 2019-11-25T09:07:33.150 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Delete the ansible users password.] ******************** 2019-11-25T09:07:33.151 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:33.190 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:33 +0000 (0:00:00.047) 0:00:04.571 ******* 2019-11-25T09:07:33.197 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:33.233 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:33 +0000 (0:00:00.042) 0:00:04.614 ******* 2019-11-25T09:07:33.652 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Ensure includedir is present in sudoers.] ************** 2019-11-25T09:07:33.654 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:07:33.690 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:33 +0000 (0:00:00.456) 0:00:05.071 ******* 2019-11-25T09:07:33.698 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:07:33.736 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:33 +0000 (0:00:00.046) 0:00:05.117 ******* 2019-11-25T09:07:35.198 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Create the cephlab_sudo sudoers.d file.] *************** changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1574672853.71-131108218147153/source", "state": "file", "uid": 0} 2019-11-25T09:07:35.241 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:35 +0000 (0:00:01.505) 0:00:06.622 ******* 2019-11-25T09:07:35.248 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "checksum": "6c8e8b7ecc895eb2094cafe75c34b58f183442a7", "dest": "/etc/sudoers.d/cephlab_sudo", "gid": 0, "group": "root", "md5sum": "c3657b9ef874d0ad278742c3210136a0", "mode": "0440", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 182, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1574672853.76-203578445173720/source", "state": "file", "uid": 0} 2019-11-25T09:07:35.291 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:35 +0000 (0:00:00.049) 0:00:06.672 ******* 2019-11-25T09:07:35.802 INFO:teuthology.task.ansible.out: TASK [ansible-managed : Add authorized keys for the ansible user.] ************* ok: [smithi153.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:35.803 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmiHwkLkoJZiQs67KfE8jf0CdXjgbbpzKywUleMk4kWC7Wdn+92q/33DXpfS8xjg7opB9GTK3EW2C/RKBpf5RdaVG/5M7tRyUDLLbS3CURUoF6aNQI+8JIuFlsIfhO3Qw4uT7NlJCqsusrSbyLtGd/Im4CzwhvP1Enn6aUfyxv2yV4HoVjXbbIp2hlRXgcW8G/9q0TMuNfX4J6dpj9/PG8BbkKH/NReL2UMtjCGyRdIQvIwJRHMozjcE21824ZatA7nBeecX0MBRS6/A286uj6Z0PgmfJXYw0yw2Z2uSWR1ptIuMEiDBt96Pzn0dvfl7g/7bCi93MBQf/6sH9n4wkN root@typica001", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:36.125 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:36.147 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:36.446 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:36.498 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:36.766 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:36.833 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:37.091 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:37.143 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:37.432 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:37.444 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZttEdz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:37.758 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:37.804 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/cm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cm", "validate_certs": true} 2019-11-25T09:07:37.922 INFO:teuthology.task.ansible.out: PLAY [teuthology] ************************************************************** skipping: no hosts matched 2019-11-25T09:07:37.932 INFO:teuthology.task.ansible.out: PLAY [testnodes] *************************************************************** 2019-11-25T09:07:38.110 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:38 +0000 (0:00:02.818) 0:00:09.491 ******* 2019-11-25T09:07:38.142 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:38 +0000 (0:00:00.031) 0:00:09.523 ******* 2019-11-25T09:07:39.026 INFO:teuthology.task.ansible.out: TASK [Gathering Facts] ********************************************************* ok: [smithi153.front.sepia.ceph.com] 2019-11-25T09:07:39.069 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] 2019-11-25T09:07:39.136 INFO:teuthology.task.ansible.out: [WARNING]: Using run_once with the free strategy is not currently supported. This task will still be executed for every host in the inventory list. 2019-11-25T09:07:39.137 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:39 +0000 (0:00:00.994) 0:00:10.517 ******* 2019-11-25T09:07:39.186 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:39 +0000 (0:00:00.044) 0:00:10.562 ******* 2019-11-25T09:07:39.625 INFO:teuthology.task.ansible.out: TASK [users : Check keys_repo HEAD sha1] *************************************** 2019-11-25T09:07:39.626 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.209577", "end": "2019-11-25 09:07:39.597400", "rc": 0, "start": "2019-11-25 09:07:39.387823", "stderr": "", "stderr_lines": [], "stdout": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "stdout_lines": ["4334b3798f2061758a627b6bdd54d0cd6b822faa"]} 2019-11-25T09:07:39.665 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:39 +0000 (0:00:00.484) 0:00:11.046 ******* 2019-11-25T09:07:39.673 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "git ls-remote https://github.com/ceph/keys HEAD | awk '{ print $1 }'", "delta": "0:00:00.210775", "end": "2019-11-25 09:07:39.618856", "rc": 0, "start": "2019-11-25 09:07:39.408081", "stderr": "", "stderr_lines": [], "stdout": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "stdout_lines": ["4334b3798f2061758a627b6bdd54d0cd6b822faa"]} 2019-11-25T09:07:39.719 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:39 +0000 (0:00:00.053) 0:00:11.100 ******* 2019-11-25T09:07:40.004 INFO:teuthology.task.ansible.out: TASK [users : Check host's /keys-repo-sha1 sentinel file] ********************** 2019-11-25T09:07:40.005 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.016569", "end": "2019-11-25 09:07:39.968133", "failed_when_result": false, "rc": 0, "start": "2019-11-25 09:07:39.951564", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-11-25T09:07:40.044 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.324) 0:00:11.425 ******* 2019-11-25T09:07:40.057 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": ["cat", "/keys-repo-sha1"], "delta": "0:00:00.013484", "end": "2019-11-25 09:07:40.025379", "failed_when_result": false, "rc": 0, "start": "2019-11-25 09:07:40.011895", "stderr": "", "stderr_lines": [], "stdout": "7b088106a1031364e6869fc2888a0e2542579008", "stdout_lines": ["7b088106a1031364e6869fc2888a0e2542579008"]} 2019-11-25T09:07:40.088 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.044) 0:00:11.469 ******* 2019-11-25T09:07:40.095 INFO:teuthology.task.ansible.out: TASK [users : Determine if we can skip users and pubkeys updates] ************** 2019-11-25T09:07:40.098 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.133 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.045) 0:00:11.514 ******* 2019-11-25T09:07:40.142 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.178 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.044) 0:00:11.559 ******* 2019-11-25T09:07:40.183 INFO:teuthology.task.ansible.out: TASK [users : Merge extra_admin_users into managed_admin_users] **************** 2019-11-25T09:07:40.185 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.224 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.045) 0:00:11.605 ******* 2019-11-25T09:07:40.237 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.279 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.054) 0:00:11.659 ******* 2019-11-25T09:07:40.283 INFO:teuthology.task.ansible.out: TASK [users : Remove managed_admin_users from managed_users] ******************* 2019-11-25T09:07:40.285 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.330 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.051) 0:00:11.711 ******* 2019-11-25T09:07:40.336 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.378 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.047) 0:00:11.759 ******* 2019-11-25T09:07:40.389 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_users list] *********************************** 2019-11-25T09:07:40.391 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.426 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.047) 0:00:11.807 ******* 2019-11-25T09:07:40.431 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.472 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.046) 0:00:11.853 ******* 2019-11-25T09:07:40.476 INFO:teuthology.task.ansible.out: TASK [users : Filter the managed_admin_users list] ***************************** 2019-11-25T09:07:40.477 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.522 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.049) 0:00:11.903 ******* 2019-11-25T09:07:40.529 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:07:40.569 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.043) 0:00:11.947 ******* 2019-11-25T09:07:40.820 INFO:teuthology.task.ansible.out: TASK [users : Set UID_MIN to 1001] ********************************************* 2019-11-25T09:07:40.821 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:07:40.858 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.292) 0:00:12.239 ******* 2019-11-25T09:07:40.887 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:07:40.928 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:07:40 +0000 (0:00:00.069) 0:00:12.309 ******* 2019-11-25T09:07:41.183 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ 2019-11-25T09:07:41.183 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-11-25T09:07:41.304 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1101, "groups": "sudo", "home": "/home/andrewschoen", "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "move_home": false, "name": "andrewschoen", "shell": "/bin/bash", "state": "present", "uid": 1101} 2019-11-25T09:07:41.496 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-11-25T09:07:41.632 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1102, "groups": "sudo", "home": "/home/zack", "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "move_home": false, "name": "zack", "shell": "/bin/bash", "state": "present", "uid": 1102} 2019-11-25T09:07:41.809 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-11-25T09:07:41.970 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1103, "groups": "sudo", "home": "/home/kdreyer", "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "move_home": false, "name": "kdreyer", "shell": "/bin/bash", "state": "present", "uid": 1103} 2019-11-25T09:07:42.139 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-11-25T09:07:42.251 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1104, "groups": "sudo", "home": "/home/adeza", "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "move_home": false, "name": "adeza", "shell": "/bin/bash", "state": "present", "uid": 1104} 2019-11-25T09:07:42.461 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-11-25T09:07:42.576 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1105, "groups": "sudo", "home": "/home/dmick", "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "move_home": false, "name": "dmick", "shell": "/bin/bash", "state": "present", "uid": 1105} 2019-11-25T09:07:42.763 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-11-25T09:07:42.872 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1106, "groups": "sudo", "home": "/home/sage", "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "move_home": false, "name": "sage", "shell": "/bin/bash", "state": "present", "uid": 1106} 2019-11-25T09:07:43.044 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-11-25T09:07:43.212 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1107, "groups": "sudo", "home": "/home/dgalloway", "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "move_home": false, "name": "dgalloway", "shell": "/bin/bash", "state": "present", "uid": 1107} 2019-11-25T09:07:43.351 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-11-25T09:07:43.545 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1108, "groups": "sudo", "home": "/home/akraitma", "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "move_home": false, "name": "akraitma", "shell": "/bin/bash", "state": "present", "uid": 1108} 2019-11-25T09:07:43.682 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-11-25T09:07:43.873 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1109, "groups": "sudo", "home": "/home/trociny", "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "move_home": false, "name": "trociny", "shell": "/bin/bash", "state": "present", "uid": 1109} 2019-11-25T09:07:43.981 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-11-25T09:07:44.174 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1110, "groups": "sudo", "home": "/home/smithfarm", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "move_home": false, "name": "smithfarm", "shell": "/bin/bash", "state": "present", "uid": 1110} 2019-11-25T09:07:44.339 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-11-25T09:07:44.506 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1111, "groups": "sudo", "home": "/home/mbenjamin", "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "move_home": false, "name": "mbenjamin", "shell": "/bin/bash", "state": "present", "uid": 1111} 2019-11-25T09:07:44.663 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-11-25T09:07:44.829 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1112, "groups": "sudo", "home": "/home/aemerson", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "move_home": false, "name": "aemerson", "shell": "/bin/bash", "state": "present", "uid": 1112} 2019-11-25T09:07:44.968 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/sbillah", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "move_home": false, "name": "sbillah", "shell": "/bin/bash", "state": "present", "uid": 1113} 2019-11-25T09:07:45.074 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1113, "groups": "sudo", "home": "/home/sbillah", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "move_home": false, "name": "sbillah", "shell": "/bin/bash", "state": "present", "uid": 1113} 2019-11-25T09:07:45.303 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-11-25T09:07:45.377 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1114, "groups": "sudo", "home": "/home/amaredia", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "move_home": false, "name": "amaredia", "shell": "/bin/bash", "state": "present", "uid": 1114} 2019-11-25T09:07:45.633 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-11-25T09:07:45.704 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1115, "groups": "sudo", "home": "/home/tserlin", "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqK 2019-11-25T09:07:45.706 INFO:teuthology.task.ansible.out:aaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "move_home": false, "name": "tserlin", "shell": "/bin/bash", "state": "present", "uid": 1115} 2019-11-25T09:07:45.957 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-11-25T09:07:46.012 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1116, "groups": "sudo", "home": "/home/dis", "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "move_home": false, "name": "dis", "shell": "/bin/bash", "state": "present", "uid": 1116} 2019-11-25T09:07:46.305 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-11-25T09:07:46.370 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1117, "groups": "sudo", "home": "/home/gregf", "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "move_home": false, "name": "gregf", "shell": "/bin/bash", "state": "present", "uid": 1117} 2019-11-25T09:07:46.613 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-11-25T09:07:46.688 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1118, "groups": "sudo", "home": "/home/joshd", "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "move_home": false, "name": "joshd", "shell": "/bin/bash", "state": "present", "uid": 1118} 2019-11-25T09:07:46.911 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-11-25T09:07:47.018 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1119, "groups": "sudo", "home": "/home/davidz", "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "move_home": false, "name": "davidz", "shell": "/bin/bash", "state": "present", "uid": 1119} 2019-11-25T09:07:47.225 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-11-25T09:07:47.330 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1120, "groups": "sudo", "home": "/home/gmeno", "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "move_home": false, "name": "gmeno", "shell": "/bin/bash", "state": "present", "uid": 1120} 2019-11-25T09:07:47.551 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-11-25T09:07:47.657 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1121, "groups": "sudo", "home": "/home/ivancich", "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "move_home": false, "name": "ivancich", "shell": "/bin/bash", "state": "present", "uid": 1121} 2019-11-25T09:07:47.874 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-11-25T09:07:47.987 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1122, "groups": "sudo", "home": "/home/wusui", "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "move_home": false, "name": "wusui", "shell": "/bin/bash", "state": "present", "uid": 1122} 2019-11-25T09:07:48.223 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-11-25T09:07:48.317 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1123, "groups": "sudo", "home": "/home/zyan", "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "move_home": false, "name": "zyan", "shell": "/bin/bash", "state": "present", "uid": 1123} 2019-11-25T09:07:48.564 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-11-25T09:07:48.670 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1124, "groups": "sudo", "home": "/home/yuriw", "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "move_home": false, "name": "yuriw", "shell": "/bin/bash", "state": "present", "uid": 1124} 2019-11-25T09:07:48.931 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-11-25T09:07:48.984 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1125, "groups": "sudo", "home": "/home/tamil", "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "move_home": false, "name": "tamil", "shell": "/bin/bash", "state": "present", "uid": 1125} 2019-11-25T09:07:49.239 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-11-25T09:07:49.324 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1126, "groups": "sudo", "home": "/home/jowilkin", "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "move_home": false, "name": "jowilkin", "shell": "/bin/bash", "state": "present", "uid": 1126} 2019-11-25T09:07:49.576 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-11-25T09:07:49.665 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1127, "groups": "sudo", "home": "/home/bhubbard", "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "move_home": false, "name": "bhubbard", "shell": "/bin/bash", "state": "present", "uid": 1127} 2019-11-25T09:07:49.917 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-11-25T09:07:50.011 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1128, "groups": "sudo", "home": "/home/yehudasa", "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "move_home": false, "name": "yehudasa", "shell": "/bin/bash", "state": "present", "uid": 1128} 2019-11-25T09:07:50.240 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-11-25T09:07:50.357 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1129, "groups": "sudo", "home": "/home/dang", "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "move_home": false, "name": "dang", "shell": "/bin/bash", "state": "present", "uid": 1129} 2019-11-25T09:07:50.576 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-11-25T09:07:50.713 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1130, "groups": "sudo", "home": "/home/branto", "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "move_home": false, "name": "branto", "shell": "/bin/bash", "state": "present", "uid": 1130} 2019-11-25T09:07:50.891 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-11-25T09:07:51.037 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1131, "groups": "sudo", "home": "/home/xiaoxichen", "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "move_home": false, "name": "xiaoxichen", "shell": "/bin/bash", "state": "present", "uid": 1131} 2019-11-25T09:07:51.207 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-11-25T09:07:51.386 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1132, "groups": "sudo", "home": "/home/ffilz", "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "move_home": false, "name": "ffilz", "shell": "/bin/bash", "state": "present", "uid": 1132} 2019-11-25T09:07:51.490 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-11-25T09:07:51.711 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1133, "groups": "sudo", "home": "/home/joao", "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "move_home": false, "name": "joao", "shell": "/bin/bash", "state": "present", "uid": 1133} 2019-11-25T09:07:51.816 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-11-25T09:07:52.051 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1134, "groups": "sudo", "home": "/home/nhm", "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "move_home": false, "name": "nhm", "shell": "/bin/bash", "state": "present", "uid": 1134} 2019-11-25T09:07:52.156 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jj", "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "move_home": false, "name": "jj", "shell": "/bin/bash", "state": "present", "uid": 1135} 2019-11-25T09:07:52.380 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1135, "groups": "sudo", "home": "/home/jj", "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "move_home": false, "name": "jj", "shell": "/bin/bash", "state": "present", "uid": 1135} 2019-11-25T09:07:52.481 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-11-25T09:07:52.745 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1136, "groups": "sudo", "home": "/home/nwatkins", "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "move_home": false, "name": "nwatkins", "shell": "/bin/bash", "state": "present", "uid": 1136} 2019-11-25T09:07:52.812 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-11-25T09:07:53.080 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1137, "groups": "sudo", "home": "/home/mkidd", "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "move_home": false, "name": "mkidd", "shell": "/bin/bash", "state": "present", "uid": 1137} 2019-11-25T09:07:53.154 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-11-25T09:07:53.429 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1138, "groups": "sudo", "home": "/home/jlopez", "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "move_home": false, "name": "jlopez", "shell": "/bin/bash", "state": "present", "uid": 1138} 2019-11-25T09:07:53.472 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-11-25T09:07:53.754 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1139, "groups": "sudo", "home": "/home/haomaiwang", "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "move_home": false, "name": "haomaiwang", "shell": "/bin/bash", "state": "present", "uid": 1139} 2019-11-25T09:07:53.777 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-11-25T09:07:54.096 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1140, "groups": "sudo", "home": "/home/jdillaman", "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "move_home": false, "name": "jdillaman", "shell": "/bin/bash", "state": "present", "uid": 1140} 2019-11-25T09:07:54.147 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-11-25T09:07:54.446 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1141, "groups": "sudo", "home": "/home/kchai", "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "move_home": false, "name": "kchai", "shell": "/bin/bash", "state": "present", "uid": 1141} 2019-11-25T09:07:54.510 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-11-25T09:07:54.788 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-11-25T09:07:54.811 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-11-25T09:07:55.081 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1143, "groups": "sudo", "home": "/home/dfuller", "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "move_home": false, "name": "dfuller", "shell": "/bin/bash", "state": "present", "uid": 1143} 2019-11-25T09:07:55.128 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-11-25T09:07:55.419 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1144, "groups": "sudo", "home": "/home/owasserm", "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "move_home": false, "name": "owasserm", "shell": "/bin/bash", "state": "present", "uid": 1144} 2019-11-25T09:07:55.472 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhishekvrshny", "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "move_home": false, "name": "abhishekvrshny", "shell": "/bin/bash", "state": "present", "uid": 1145} 2019-11-25T09:07:55.730 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1145, "groups": "sudo", "home": "/home/abhishekvrshny", "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "move_home": false, "name": "abhishekvrshny", "shell": "/bin/bash", "state": "present", "uid": 1145} 2019-11-25T09:07:55.823 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-11-25T09:07:56.060 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1146, "groups": "sudo", "home": "/home/vasu", "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "move_home": false, "name": "vasu", "shell": "/bin/bash", "state": "present", "uid": 1146} 2019-11-25T09:07:56.157 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-11-25T09:07:56.348 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1147, "groups": "sudo", "home": "/home/smohan", "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "move_home": false, "name": "smohan", "shell": "/bin/bash", "state": "present", "uid": 1147} 2019-11-25T09:07:56.507 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-11-25T09:07:56.688 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1148, "groups": "sudo", "home": "/home/abhi", "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "move_home": false, "name": "abhi", "shell": "/bin/bash", "state": "present", "uid": 1148} 2019-11-25T09:07:56.854 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/shehbazj", "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "move_home": false, "name": "shehbazj", "shell": "/bin/bash", "state": "present", "uid": 1149} 2019-11-25T09:07:57.021 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1149, "groups": "sudo", "home": "/home/shehbazj", "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "move_home": false, "name": "shehbazj", "shell": "/bin/bash", "state": "present", "uid": 1149} 2019-11-25T09:07:57.177 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-11-25T09:07:57.344 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1150, "groups": "sudo", "home": "/home/cbodley", "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "move_home": false, "name": "cbodley", "shell": "/bin/bash", "state": "present", "uid": 1150} 2019-11-25T09:07:57.502 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/fche", "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "move_home": false, "name": "fche", "shell": "/bin/bash", "state": "present", "uid": 1151} 2019-11-25T09:07:57.690 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1151, "groups": "sudo", "home": "/home/fche", "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "move_home": false, "name": "fche", "shell": "/bin/bash", "state": "present", "uid": 1151} 2019-11-25T09:07:57.825 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/onyb", "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "move_home": false, "name": "onyb", "shell": "/bin/bash", "state": "present", "uid": 1152} 2019-11-25T09:07:58.015 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1152, "groups": "sudo", "home": "/home/onyb", "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "move_home": false, "name": "onyb", "shell": "/bin/bash", "state": "present", "uid": 1152} 2019-11-25T09:07:58.170 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-11-25T09:07:58.329 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1153, "groups": "sudo", "home": "/home/mwatts", "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "move_home": false, "name": "mwatts", "shell": "/bin/bash", "state": "present", "uid": 1153} 2019-11-25T09:07:58.513 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/oprypin", "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "move_home": false, "name": "oprypin", "shell": "/bin/bash", "state": "present", "uid": 1154} 2019-11-25T09:07:58.639 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1154, "groups": "sudo", "home": "/home/oprypin", "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "move_home": false, "name": "oprypin", "shell": "/bin/bash", "state": "present", "uid": 1154} 2019-11-25T09:07:58.827 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-11-25T09:07:58.973 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-11-25T09:07:59.161 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-11-25T09:07:59.287 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1156, "groups": "sudo", "home": "/home/pdonnell", "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "move_home": false, "name": "pdonnell", "shell": "/bin/bash", "state": "present", "uid": 1156} 2019-11-25T09:07:59.485 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-11-25T09:07:59.611 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1157, "groups": "sudo", "home": "/home/jlayton", "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "move_home": false, "name": "jlayton", "shell": "/bin/bash", "state": "present", "uid": 1157} 2019-11-25T09:07:59.823 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-11-25T09:07:59.959 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1158, "groups": "sudo", "home": "/home/rzarzynski", "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "move_home": false, "name": "rzarzynski", "shell": "/bin/bash", "state": "present", "uid": 1158} 2019-11-25T09:08:00.114 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-11-25T09:08:00.303 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1159, "groups": "sudo", "home": "/home/rdias", "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "move_home": false, "name": "rdias", "shell": "/bin/bash", "state": "present", "uid": 1159} 2019-11-25T09:08:00.450 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/asheplyakov", "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "move_home": false, "name": "asheplyakov", "shell": "/bin/bash", "state": "present", "uid": 1160} 2019-11-25T09:08:00.621 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1160, "groups": "sudo", "home": "/home/asheplyakov", "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "move_home": false, "name": "asheplyakov", "shell": "/bin/bash", "state": "present", "uid": 1160} 2019-11-25T09:08:00.754 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-11-25T09:08:00.961 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1161, "groups": "sudo", "home": "/home/vshankar", "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "move_home": false, "name": "vshankar", "shell": "/bin/bash", "state": "present", "uid": 1161} 2019-11-25T09:08:01.096 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-11-25T09:08:01.304 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1162, "groups": "sudo", "home": "/home/akupczyk", "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "move_home": false, "name": "akupczyk", "shell": "/bin/bash", "state": "present", "uid": 1162} 2019-11-25T09:08:01.407 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-11-25T09:08:01.631 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1163, "groups": "sudo", "home": "/home/nojha", "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "move_home": false, "name": "nojha", "shell": "/bin/bash", "state": "present", "uid": 1163} 2019-11-25T09:08:01.731 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-11-25T09:08:01.950 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1164, "groups": "sudo", "home": "/home/ifed01", "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "move_home": false, "name": "ifed01", "shell": "/bin/bash", "state": "present", "uid": 1164} 2019-11-25T09:08:02.096 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-11-25T09:08:02.276 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1165, "groups": "sudo", "home": "/home/myoungwon", "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "move_home": false, "name": "myoungwon", "shell": "/bin/bash", "state": "present", "uid": 1165} 2019-11-25T09:08:02.455 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/jwilliamson", "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "move_home": false, "name": "jwilliamson", "shell": "/bin/bash", "state": "present", "uid": 1166} 2019-11-25T09:08:02.608 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1166, "groups": "sudo", "home": "/home/jwilliamson", "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "move_home": false, "name": "jwilliamson", "shell": "/bin/bash", "state": "present", "uid": 1166} 2019-11-25T09:08:02.801 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-11-25T09:08:02.931 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1167, "groups": "sudo", "home": "/home/gabrioux", "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "move_home": false, "name": "gabrioux", "shell": "/bin/bash", "state": "present", "uid": 1167} 2019-11-25T09:08:03.129 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-11-25T09:08:03.258 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1168, "groups": "sudo", "home": "/home/leseb", "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "move_home": false, "name": "leseb", "shell": "/bin/bash", "state": "present", "uid": 1168} 2019-11-25T09:08:03.458 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-11-25T09:08:03.595 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1169, "groups": "sudo", "home": "/home/hchen", "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "move_home": false, "name": "hchen", "shell": "/bin/bash", "state": "present", "uid": 1169} 2019-11-25T09:08:03.793 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-11-25T09:08:03.942 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1170, "groups": "sudo", "home": "/home/jcollin", "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "move_home": false, "name": "jcollin", "shell": "/bin/bash", "state": "present", "uid": 1170} 2019-11-25T09:08:04.155 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-11-25T09:08:04.294 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1171, "groups": "sudo", "home": "/home/xxg", "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "move_home": false, "name": "xxg", "shell": "/bin/bash", "state": "present", "uid": 1171} 2019-11-25T09:08:04.476 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-11-25T09:08:04.640 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1172, "groups": "sudo", "home": "/home/pcuzner", "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "move_home": false, "name": "pcuzner", "shell": "/bin/bash", "state": "present", "uid": 1172} 2019-11-25T09:08:04.819 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/liupan", "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "move_home": false, "name": "liupan", "shell": "/bin/bash", "state": "present", "uid": 1173} 2019-11-25T09:08:04.991 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1173, "groups": "sudo", "home": "/home/liupan", "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "move_home": false, "name": "liupan", "shell": "/bin/bash", "state": "present", "uid": 1173} 2019-11-25T09:08:05.147 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-11-25T09:08:05.330 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1174, "groups": "sudo", "home": "/home/mkogan", "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "move_home": false, "name": "mkogan", "shell": "/bin/bash", "state": "present", "uid": 1174} 2019-11-25T09:08:05.456 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/amarangone", "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "move_home": false, "name": "amarangone", "shell": "/bin/bash", "state": "present", "uid": 1175} 2019-11-25T09:08:05.636 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1175, "groups": "sudo", "home": "/home/amarangone", "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "move_home": false, "name": "amarangone", "shell": "/bin/bash", "state": "present", "uid": 1175} 2019-11-25T09:08:05.803 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-11-25T09:08:05.993 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1142, "groups": "sudo", "home": "/home/vumrao", "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "move_home": false, "name": "vumrao", "shell": "/bin/bash", "state": "present", "uid": 1142} 2019-11-25T09:08:06.155 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/kmroz", "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "move_home": false, "name": "kmroz", "shell": "/bin/bash", "state": "present", "uid": 1176} 2019-11-25T09:08:06.323 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1176, "groups": "sudo", "home": "/home/kmroz", "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "move_home": false, "name": "kmroz", "shell": "/bin/bash", "state": "present", "uid": 1176} 2019-11-25T09:08:06.483 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-11-25T09:08:06.671 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1177, "groups": "sudo", "home": "/home/henrix", "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "move_home": false, "name": "henrix", "shell": "/bin/bash", "state": "present", "uid": 1177} 2019-11-25T09:08:06.824 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-11-25T09:08:06.972 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1178, "groups": "sudo", "home": "/home/pbs1108", "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "move_home": false, "name": "pbs1108", "shell": "/bin/bash", "state": "present", "uid": 1178} 2019-11-25T09:08:07.147 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-11-25T09:08:07.316 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1179, "groups": "sudo", "home": "/home/clacroix", "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "move_home": false, "name": "clacroix", "shell": "/bin/bash", "state": "present", "uid": 1179} 2019-11-25T09:08:07.454 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-11-25T09:08:07.642 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1180, "groups": "sudo", "home": "/home/epuertat", "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "move_home": false, "name": "epuertat", "shell": "/bin/bash", "state": "present", "uid": 1180} 2019-11-25T09:08:07.748 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-11-25T09:08:07.983 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1181, "groups": "sudo", "home": "/home/tdehler", "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "move_home": false, "name": "tdehler", "shell": "/bin/bash", "state": "present", "uid": 1181} 2019-11-25T09:08:08.105 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-11-25T09:08:08.329 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1182, "groups": "sudo", "home": "/home/laura", "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "move_home": false, "name": "laura", "shell": "/bin/bash", "state": "present", "uid": 1182} 2019-11-25T09:08:08.396 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/adamyanova", "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "move_home": false, "name": "adamyanova", "shell": "/bin/bash", "state": "present", "uid": 1183} 2019-11-25T09:08:08.693 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1183, "groups": "sudo", "home": "/home/adamyanova", "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "move_home": false, "name": "adamyanova", "shell": "/bin/bash", "state": "present", "uid": 1183} 2019-11-25T09:08:08.726 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-11-25T09:08:09.007 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1184, "groups": "sudo", "home": "/home/yaarit", "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "move_home": false, "name": "yaarit", "shell": "/bin/bash", "state": "present", "uid": 1184} 2019-11-25T09:08:09.060 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-11-25T09:08:09.382 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1185, "groups": "sudo", "home": "/home/rpavani1998", "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "move_home": false, "name": "rpavani1998", "shell": "/bin/bash", "state": "present", "uid": 1185} 2019-11-25T09:08:09.399 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-11-25T09:08:09.704 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1186, "groups": "sudo", "home": "/home/rishabh", "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "move_home": false, "name": "rishabh", "shell": "/bin/bash", "state": "present", "uid": 1186} 2019-11-25T09:08:09.706 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-11-25T09:08:10.003 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1187, "groups": "sudo", "home": "/home/skrah", "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "move_home": false, "name": "skrah", "shell": "/bin/bash", "state": "present", "uid": 1187} 2019-11-25T09:08:10.016 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-11-25T09:08:10.346 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1188, "groups": "sudo", "home": "/home/smanjara", "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "move_home": false, "name": "smanjara", "shell": "/bin/bash", "state": "present", "uid": 1188} 2019-11-25T09:08:10.379 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-11-25T09:08:10.695 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1189, "groups": "sudo", "home": "/home/bengland", "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "move_home": false, "name": "bengland", "shell": "/bin/bash", "state": "present", "uid": 1189} 2019-11-25T09:08:10.707 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-11-25T09:08:11.009 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1190, "groups": "sudo", "home": "/home/pnawracay", "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "move_home": false, "name": "pnawracay", "shell": "/bin/bash", "state": "present", "uid": 1190} 2019-11-25T09:08:11.066 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-11-25T09:08:11.338 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1191, "groups": "sudo", "home": "/home/alfonsomthd", "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "move_home": false, "name": "alfonsomthd", "shell": "/bin/bash", "state": "present", "uid": 1191} 2019-11-25T09:08:11.439 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-11-25T09:08:11.667 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1192, "groups": "sudo", "home": "/home/oliveiradan", "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "move_home": false, "name": "oliveiradan", "shell": "/bin/bash", "state": "present", "uid": 1192} 2019-11-25T09:08:11.804 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-11-25T09:08:12.022 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1193, "groups": "sudo", "home": "/home/swagner", "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "move_home": false, "name": "swagner", "shell": "/bin/bash", "state": "present", "uid": 1193} 2019-11-25T09:08:12.148 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-11-25T09:08:12.360 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1194, "groups": "sudo", "home": "/home/yuvalif", "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "move_home": false, "name": "yuvalif", "shell": "/bin/bash", "state": "present", "uid": 1194} 2019-11-25T09:08:12.425 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-11-25T09:08:12.707 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1195, "groups": "sudo", "home": "/home/kkeithle", "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "move_home": false, "name": "kkeithle", "shell": "/bin/bash", "state": "present", "uid": 1195} 2019-11-25T09:08:12.750 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-11-25T09:08:13.000 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1196, "groups": "sudo", "home": "/home/emmericp", "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "move_home": false, "name": "emmericp", "shell": "/bin/bash", "state": "present", "uid": 1196} 2019-11-25T09:08:13.074 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-11-25T09:08:13.314 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1197, "groups": "sudo", "home": "/home/mchangir", "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "move_home": false, "name": "mchangir", "shell": "/bin/bash", "state": "present", "uid": 1197} 2019-11-25T09:08:13.394 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-11-25T09:08:13.681 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1198, "groups": "sudo", "home": "/home/sidharthanup", "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "move_home": false, "name": "sidharthanup", "shell": "/bin/bash", "state": "present", "uid": 1198} 2019-11-25T09:08:13.713 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-11-25T09:08:14.018 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1199, "groups": "sudo", "home": "/home/varsha", "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "move_home": false, "name": "varsha", "shell": "/bin/bash", "state": "present", "uid": 1199} 2019-11-25T09:08:14.063 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-11-25T09:08:14.319 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1200, "groups": "sudo", "home": "/home/sjust", "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "move_home": false, "name": "sjust", "shell": "/bin/bash", "state": "present", "uid": 1200} 2019-11-25T09:08:14.388 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-11-25T09:08:14.649 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1201, "groups": "sudo", "home": "/home/ideepika", "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "move_home": false, "name": "ideepika", "shell": "/bin/bash", "state": "present", "uid": 1201} 2019-11-25T09:08:14.747 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-11-25T09:08:15.002 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1202, "groups": "sudo", "home": "/home/gsalomon", "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "move_home": false, "name": "gsalomon", "shell": "/bin/bash", "state": "present", "uid": 1202} 2019-11-25T09:08:15.044 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-11-25T09:08:15.346 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1203, "groups": "sudo", "home": "/home/soumyakoduri", "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "move_home": false, "name": "soumyakoduri", "shell": "/bin/bash", "state": "present", "uid": 1203} 2019-11-25T09:08:15.368 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-11-25T09:08:15.639 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1204, "groups": "sudo", "home": "/home/kyr", "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "move_home": false, "name": "kyr", "shell": "/bin/bash", "state": "present", "uid": 1204} 2019-11-25T09:08:15.694 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-11-25T09:08:15.980 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1205, "groups": "sudo", "home": "/home/sseshasa", "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "move_home": false, "name": "sseshasa", "shell": "/bin/bash", "state": "present", "uid": 1205} 2019-11-25T09:08:16.024 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-11-25T09:08:16.337 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1206, "groups": "sudo", "home": "/home/rfriedma", "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "move_home": false, "name": "rfriedma", "shell": "/bin/bash", "state": "present", "uid": 1206} 2019-11-25T09:08:16.361 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-11-25T09:08:16.675 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1207, "groups": "sudo", "home": "/home/lmb", "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "move_home": false, "name": "lmb", "shell": "/bin/bash", "state": "present", "uid": 1207} 2019-11-25T09:08:16.686 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-11-25T09:08:17.003 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-11-25T09:08:17.005 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1208, "groups": "sudo", "home": "/home/ksirivad", "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "move_home": false, "name": "ksirivad", "shell": "/bin/bash", "state": "present", "uid": 1208} 2019-11-25T09:08:17.323 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-11-25T09:08:17.324 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1209, "groups": "sudo", "home": "/home/ofriedma", "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "move_home": false, "name": "ofriedma", "shell": "/bin/bash", "state": "present", "uid": 1209} 2019-11-25T09:08:17.648 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1210, "groups": "sudo", "home": "/home/h4iresis", "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "move_home": false, "name": "h4iresis", "shell": "/bin/bash", "state": "present", "uid": 1210} 2019-11-25T09:08:17.660 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-11-25T09:08:17.983 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1211, "groups": "sudo", "home": "/home/shyukri", "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "move_home": false, "name": "shyukri", "shell": "/bin/bash", "state": "present", "uid": 1211} 2019-11-25T09:08:18.008 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-11-25T09:08:18.311 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1212, "groups": "sudo", "home": "/home/scarvalhojr", "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "move_home": false, "name": "scarvalhojr", "shell": "/bin/bash", "state": "present", "uid": 1212} 2019-11-25T09:08:18.312 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-11-25T09:08:18.629 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1213, "groups": "sudo", "home": "/home/rakesh", "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "move_home": false, "name": "rakesh", "shell": "/bin/bash", "state": "present", "uid": 1213} 2019-11-25T09:08:18.940 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1214, "groups": "sudo", "home": "/home/ryg-", "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "move_home": false, "name": "ryg-", "shell": "/bin/bash", "state": "present", "uid": 1214} 2019-11-25T09:08:18.963 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-11-25T09:08:19.263 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 1155, "groups": "sudo", "home": "/home/prsrivas", "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "move_home": false, "name": "prsrivas", "shell": "/bin/bash", "state": "present", "uid": 1155} 2019-11-25T09:08:19.296 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-11-25T09:08:19.611 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "append": true, "changed": false, "comment": "", "group": 10102, "groups": "sudo", "home": "/home/zdover", "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "move_home": false, "name": "zdover", "shell": "/bin/bash", "state": "present", "uid": 10102} 2019-11-25T09:08:19.969 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-11-25T09:08:20.099 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:20 +0000 (0:00:39.171) 0:00:51.480 ******* 2019-11-25T09:08:20.126 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-11-25T09:08:20.163 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:20 +0000 (0:00:00.062) 0:00:51.543 ******* 2019-11-25T09:08:20.211 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-11-25T09:08:20.215 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-11-25T09:08:20.217 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04 2019-11-25T09:08:20.219 INFO:teuthology.task.ansible.out:okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "jos 2019-11-25T09:08:20.220 INFO:teuthology.task.ansible.out:hd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jo 2019-11-25T09:08:20.221 INFO:teuthology.task.ansible.out:wilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f 2019-11-25T09:08:20.223 INFO:teuthology.task.ansible.out:966b2794c94caa110d313e927a1c1b\n"}, {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h 2019-11-25T09:08:20.224 INFO:teuthology.task.ansible.out:7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6 2019-11-25T09:08:20.225 INFO:teuthology.task.ansible.out:c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa 2019-11-25T09:08:20.227 INFO:teuthology.task.ansible.out:2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db2 2019-11-25T09:08:20.228 INFO:teuthology.task.ansible.out:93dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.red 2019-11-25T09:08:20.232 INFO:teuthology.task.ansible.out:hat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f 2019-11-25T09:08:20.233 INFO:teuthology.task.ansible.out:57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-11-25T09:08:20.268 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:20 +0000 (0:00:00.105) 0:00:51.649 ******* 2019-11-25T09:08:20.319 INFO:teuthology.task.ansible.out: TASK [users : Create all admin users with sudo access.] ************************ changed: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": "", "create_home": true, "group": 10104, "groups": "sudo", "home": "/home/rraja", "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "name": "rraja", "shell": "/bin/bash", "state": "present", "system": false, "uid": 10103} 2019-11-25T09:08:20.591 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:20 +0000 (0:00:00.323) 0:00:51.972 ******* 2019-11-25T09:08:20.619 INFO:teuthology.task.ansible.out: TASK [users : Create all users without sudo access.] *************************** 2019-11-25T09:08:20.654 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:20 +0000 (0:00:00.063) 0:00:52.035 ******* 2019-11-25T09:08:20.702 INFO:teuthology.task.ansible.out: TASK [users : Merge managed_users and managed_admin users] ********************* 2019-11-25T09:08:20.707 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"pubkey_users": [{"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, {"name": "tr 2019-11-25T09:08:20.708 INFO:teuthology.task.ansible.out:ociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04 2019-11-25T09:08:20.710 INFO:teuthology.task.ansible.out:okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, {"name": "jos 2019-11-25T09:08:20.711 INFO:teuthology.task.ansible.out:hd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, {"name": "jowilkin", "ovpn": "jo 2019-11-25T09:08:20.712 INFO:teuthology.task.ansible.out:wilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851 2019-11-25T09:08:20.714 INFO:teuthology.task.ansible.out:f7f9e1a255327d6"}, {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde189 2019-11-25T09:08:20.715 INFO:teuthology.task.ansible.out:0af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f45903 2019-11-25T09:08:20.717 INFO:teuthology.task.ansible.out:0cb7a195"}, {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d7 2019-11-25T09:08:20.718 INFO:teuthology.task.ansible.out:12ba2"}, {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006" 2019-11-25T09:08:20.720 INFO:teuthology.task.ansible.out:}, {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339 2019-11-25T09:08:20.721 INFO:teuthology.task.ansible.out:054088d257a16cf9fafcfe05b27fa678e"}, {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b 2019-11-25T09:08:20.723 INFO:teuthology.task.ansible.out:02ae1a2c43dac1bbcd1b538e70312189"}, {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69 2019-11-25T09:08:20.724 INFO:teuthology.task.ansible.out:088eda5efc"}, {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}]}, "changed": false} 2019-11-25T09:08:20.766 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:20 +0000 (0:00:00.111) 0:00:52.147 ******* 2019-11-25T09:08:21.185 INFO:teuthology.task.ansible.out: TASK [users : Clone the keys repo] ********************************************* 2019-11-25T09:08:21.187 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com -> localhost] => {"after": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "attempts": 1, "before": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "changed": false, "remote_url_changed": false} 2019-11-25T09:08:21.224 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:21 +0000 (0:00:00.458) 0:00:52.605 ******* 2019-11-25T09:08:21.396 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com -> localhost] => {"after": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "attempts": 1, "before": "4334b3798f2061758a627b6bdd54d0cd6b822faa", "changed": false, "remote_url_changed": false} 2019-11-25T09:08:21.436 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:21 +0000 (0:00:00.210) 0:00:52.816 ******* 2019-11-25T09:08:21.618 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-11-25T09:08:21.816 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDehvF7TbkfX9Ga4fNdq/j9GHu3c9cNk4iIHVs7e14j0h9DlV14aPyn/M0+z90PK+5p2uL9EvRzkbsPrfxpG3pxUa1pmVmKIPvBruA7ln8tExOVwO3XEsBG7b8VWUMlQmwRS9WEfkKAIHlr9cgIFGb1yrMW30RyL7keO1giotAq/wR9pMicyqYAbZOtI3MuPsDVTNxtyD3RwRUQicz6vNaK/RwfMCZVgdvoinrJ7VSyFVEiitzuQqMlrF935sfLvhuyxmwvJKfZI8QMEYcFbKd1cDnCk9n/PySNRY96gb41nb5pUk+0eaDu37jnmj7LLqoidD3UWUuHNnGBAAk+zNlD andrew.schoen@gmail.com", "key_options": null, "keyfile": "/home/andrewschoen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "andrewschoen", "validate_certs": true} 2019-11-25T09:08:21.983 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-11-25T09:08:22.171 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBwJCoMbbxQrI9jM5I/lX0MYDB57LzYXDJTXqmLfJ8E5IHwQbn708/EFXAHuJlCyoJAuA8FaU/Y5/l2LBlVXwp9v+8ftk/D6AcsuQ+Hi0ZLCTvW10OUIS2cwX/of//JJIu6roy3r3SWwUcQvwjaZYLm7QkC+1StsLNMHdgiVRFiibqml72Q53VTtSTt6z2aXjrHzhpZq0hBK/13aknzAjHvGprQnchKVzhA/7A5pArF3CXpnI7aCwKvvjU0AcvqqD8WNLMpF4hFSoTK7xluKB+5UAJ36wefIO2KFM/zMwUtl4/aDhDdPeSaQRWq/q7viKLVNS0X9j2PjSRTfYhJNZr zack@zack-cerzas-macbook-pro.local", "key_options": null, "keyfile": "/home/zack/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zack", "validate_certs": true} 2019-11-25T09:08:22.296 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-11-25T09:08:22.515 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4eHmz10szeHNS3dNejKokW85ksB+iR4HGOFsmQM11Ni68Nm5aqEKvkOZU8TpY92vpCQL0A68GlrXB845cACdyk6HUJYyNNNMC43l1FYWOwjMqQBSdj8W3VQDTA6eiG60mt5fgI8fyR38rKzIA1MnTBkSSjuh5kQVJ9bdEp3GuY5oc8vxDNBlGJ6LYnyEWt/pqL2J+mpjqnOjsC+EbE2exhP9O+mvzpQiyo/+dEN1COwX3//pNRXGfOSeOczHNsJE8Eu+j/n/BlW57++sJyFMkzS7bUxMSGM6quvjQZ7RT1c5JM6vLEiQyzQxoRgzY93h1yKlOstBi0NamtpqHQZGP kdreyer@mbp.ktdreyer.com", "key_options": null, "keyfile": "/home/kdreyer/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kdreyer", "validate_certs": true} 2019-11-25T09:08:22.629 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-11-25T09:08:22.871 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCWy2i6Iv6IWLh53Z44O4fQ0ctxR8odCAWIZAaSQXAxxC++RS+95Zr+BZ5wJwsBOyP1G4AAhl+9uwnn3NmDDG7fKGexPoQrTZZ27KdJ3bHpZRq5LJf9byWFwz4rDsqIqkGjI9sUkfa6nV7ci2BdEkIgLupTT2uXUzTb/njOjI7Fc+cQ7ac4QfJ+yeIcqxeje/YYDF3068V4z1yynk9q2KRXYdERKQghaWDNJlbdBhn/fmsam6okOtUnsXjE88qGPd+LVHWfK0EfYvYXiAoOaJDK8d4EXnJ4RKJceQPBk8ravuQtKVwnRIr8WtmILHuoztthfwqb2WvZ5iMDt6v7oQZ alfredo@papaya.local", "key_options": null, "keyfile": "/home/adeza/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adeza", "validate_certs": true} 2019-11-25T09:08:22.924 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-11-25T09:08:22.926 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-11-25T09:08:23.216 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC305rmql/Qm8Si+DFvAs77hSUZakwiXhhAhSetS89bWq3AdPQTL1MUXJHFm3AMzwzb00tndvsl1AUmFseXLfAi87daNcKJdwEj/JvXgcHKzAXOpVRfArHh08JnOumQiP4EoLYvq7rL+3LjEFURKOWJLLnGxh4E+iiJAnxLTG3WuUcnz9kXVTwtmIs60+2yH+gZJkkB04mnivPYjX46sHigoKgtlE2JnQfOU1lBkeZaBeUhoZX2rIKwRyGELPPcUBwHk10izyk6tePLAw9bSWeS9uEaa/m+qa8neJDyR4/hWQNR9yRXp0YjKkiKoFog+qtsBt3xBNzTPsFLxZ5zugmp dmick@angus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/onMoOCUhkp+UiJAq+ye4P70sfeV+uMhgWKLfLBh/1SYq1xxb9bSmjBnETO/9KvukRhzlReBQ7y+BKRhm037P7LuTUY4r0Y8JB6BXkjjCY1yXZJC6kvB7xFsvyiDum5rrdeWMQ/Rhh7YJpze5KY1nHLkyU9X7Ewak0kLEyduJEKekbsOom6l0wZ2ZYrBEKFWJXUDk5icUuWWg3mhKaKnLNWVg1lUw8jzYTJYfjR9GUgzZWXtzkAaI3ugpLOlQ+KEls03/WAFcWA9F9ExqzIsJpvEks27LeFBGVE3PjVNKMqqBZQlZtIEz1iguWJaWMi5lDIfXkvVOFabtIpX/BZRf dmick@dmick-MacBookAir", "key_options": null, "keyfile": "/home/dmick/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dmick", "va 2019-11-25T09:08:23.218 INFO:teuthology.task.ansible.out:lidate_certs": true} 2019-11-25T09:08:23.280 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-11-25T09:08:23.545 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJ4z0gLwjexgyixdTuiwaxpPnq7w4FOKxHsQKKAHTlKRu11ynB+9APbtqaAYss8yDqHaHyewPdE+9JMYhZX8rUgcKv2H7jBCQ/Enfl0UyY643VQnuG8lr2Ipk6ELuL5NuuTb7Npo4zKyq/0649kPD46k/X/qLuKzhN49k50S3XJq5WwHi6CJvR8DzPOkjied0o1BHpfcCCIMorUcyfVxqmeu2m+uSE15ncQsd8BIvH4/YNoncUGYxqVhGB+FNUlWdhV0SPRW7V/gmyvFW5qdYX2QhNZAQHChdEH1E9AI7WiCFkyLRx39qYgXgqj8vOOnlwHgy/oJnpnu/Mlvj1uETX sage@montreal\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCVPfoCpv2/9uWCwEo4clWt3i1O8ZPpB2FcQbBLxY3iLQWZJDe74rNWPBj0cJVCytTHxYV0e5HxYyChOG60ca5odHlPNB+/Z4WLU9LVuHiCYsElivJlwvD+l9kzI+UMGJb3HCHqx2yIRGXpEkSMFR1+w5Oz9sInPD7JJmO8Hb9KkoeqZtkJL0aic/M5bDWBut6SSfYlmh8qnQYbkCDiXg/KtOcy1ZJXQ9ZLjsCbpSzeBNbdUOxEQ1+5vUe00vHpl0vOKtRrzgTne2ct1IxbSjdL16ZgHi5BQAREXc9UUG3z7NPOmnwkuKkkpyLmwcOusot5tYHrxks+5i6EoGEJrvJ3 sage@maetl", "key_options": null, "keyfile": "/home/sage/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sage", "validate_certs": true} 2019-11-25T09:08:23.567 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-11-25T09:08:23.569 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} 2019-11-25T09:08:23.899 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBAN3MlkYQNsm1uB6uJQ7hWJ1Ro9Gusrhi/b/KshPMFh51VWBQn71LdCCjdrsE6FWs5FYE/hr7MITJe+si7aZCmuf8U9tc+oiYh3spXtRhJVIM80FOVRXUmt2XR8I+oNbpcFDayYu956Volafksk3oYjsO356swj7+Q+AUQVF4bDJRAAAAFQCkB2YZKItSDL7XfAl6ZmJ21vqUMwAAAIEAsZspNIUwssVJGO4r3M3BECWklWLEFviIjGVKdPWulN10eRCczZrH7vMOs8coHKDQqYdS2RejlojT03FjFAqeBiTeAlcmgHoev+vqTg2Yhqx3gROzNc6NmG887owCOU4j7LJQP6SX8I8EpSXCMNOZliv2ZokjX2DuRsuIDxQdc84AAACBAMAp++fcqmGS7x7ic2x4rhyiYmdqlMIl8lrM3GkVR6qGuIVBvOGl38ieJKcTBLeRgyC04uIMfk8P9lLvZw3UF97qH3V0imKfq8aYnBGxCv+IjtdOrR5k4hrUW+P0qipvU+4dgHlpqwTKB4MtD9uDxvhENWvqcCnOS5rI6dddaZT6 dgallowa@dgallowa.csb\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKT1KilDopHJzDZ90QJx4+7WVIC4WNdvpD6VrS83B3PFOplNyXZLfDSDRZ4wEzIKLwtR9vLWWInVBNFU00W/2dObSC+l9GDzKsHtO3KSVmAjHEk6ZAJ5Mc3eUPR0lAywpFD8+yg+kDDIkRtCtoHDq+kFvNQutdvrMuhNWOZLyQ2N2XrpRJVD2kiJvaml6nQoZXMtVQGPVcYbn3TRed/wxT/han2u6j/wiElv4r9ty+9zi3P1MoNiALD53sD7wMr/5Si6PX+5zWcXTNcQmTuGpiLwGw0RxCwr4gtwqKBaNae2d1bwnHanLc1KvQEw29O6aBca2f7GWxFY6QtQV/mNft dgalloway@w541\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEI+g74GcJe/03nDypMzDWqg93K1h13xQPFvCIN7ElNZ+ulY5av5KLnv/gmLbYGRPlgxC7LNZXE7YQPUo2DfA8awnDkAkE/nWvL4fQjUxmaiPthjIWsKvk9EvPIqzLjDbwsiLiRPE0Sxka0qBa1GHU1QYN6cw7l9nWF0dG/TsKw+QmuArBKMgm9Th/QVLq1ARHC/07M1n195QvBjdrKSzOUl+2L1tADHg++sEWDq/MJMiylyNdxp79Wz7aTIlR6l8NV+Ega9LUb6B6xmO7k+J+/hJF7gYOeFl+YqnZtt 2019-11-25T09:08:23.901 INFO:teuthology.task.ansible.out:Edz+o8FbLjhTjOF9QkBsGWlrawXtQln/cVCryV+LAt57L dgalloway@p50", "key_options": null, "keyfile": "/home/dgalloway/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dgalloway", "validate_certs": true} ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-11-25T09:08:24.244 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDo3gswIiZ2xPWMUQNjV1Gd4gYSxVpJM8Jb5pH1lqvoKAsmhVI30qTYxIaezzDWfSYwdkNYQAC9eM5v7EMQrOI7eZtTzCH+gbF79GciGCLL2bDze1I8hJ+jUu4bv+wnmvKsHyyOVBBJPSJ+eGFCzsu7vOiK4iFX0zgCXYwvP+A4Y/+kVR1en39gMtD6Hrw8ACBhxt+C2G2zc1807NRj3pIw1GvQILa9Sm3VKdNSrHzGB+MBooUoRmmjjzZ/bRwHPWVj+sX+xkoX4P2zwUcfMgZ/f4GgSgHyEvS3oxqbSaZp0pidZyoCaLRgqVPiUgXQEm0HA5PDDULAthQOfxQveKB/ akraitma@aklap", "key_options": null, "keyfile": "/home/akraitma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akraitma", "validate_certs": true} 2019-11-25T09:08:24.246 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-11-25T09:08:24.257 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.579 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDSc2rWtLtZM0CBWwaxhSfd0T++ZTX9c8kQrYT0rsQlJ6Hxj4c3+ypwh4iivdtFpyz4clxWkb/wimi48PvjNyZdudB9BeQUBeFfLiro3LnFEbKhM1RwRI89s34mRwmC0LLUZwcdX80zc6/fKuPzvBzR1PiY9tWRyIpYErQAHa2jEOE2MpFDHnlbXdfXFh5Tw3OmC1qBlQvgcrivA9lVaVlf+hfwmOQwXFIG3PCJLNlXDshCSUGAfYKBRvQaKS8sOf9rQaflPrwqJ8hDgVVQkpp69yVsGtahbBmjs6QS6C8EkGUFQmvrIcrQ8+5mRlRAmK2DMTP3cAcsYcMOJ+9KwP+H mgolub@zhuzha", "key_options": null, "keyfile": "/home/trociny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "trociny", "validate_certs": true} 2019-11-25T09:08:24.590 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.612 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-11-25T09:08:24.613 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-11-25T09:08:24.623 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.634 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.645 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.656 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.931 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANarZ+C62DPffRIhaCuCtXLChrfqwXgWqGHOLLkmPTtZ3/EE8ZiN+LY1ZcmP939AxXi8Oe+9LepwAdsoQKZctlC7XToyhjIk+rVSPyn3zl3BTC2AxfCKq2vqj65a1sc35Whrw7OaUa2fr5iHSazP6HuG01DhJ9l8HJsLKmiwWChZAAAAFQCNmunc7BNLWrqyrYwINTfscowkbQAAAIEAyYERcunZYJAZ9Eno18kM+ylq0jcQmnPqyIvssWXTttUZnaGQvbVxABeyr6ZKuUyZQfyetofquOGBKPBuzV/7posl5ShDuAcxbS1zhrmk9glTA1yFLvznDdWdLOWG06tjOc78G2AmldKHCdPsmgh2D7WyRr4QY4rcCvwKfZ6+eEYAAACBAJZrOlOlygfPZ4ZB0+EbdYACz1+L/iYNNAOBp51TRutNceLVrNJNOVdmCMLuF0A1go4kPSD/05aJOkOyW86gYLA5DXaW3961+kPInOFAe8OTyQYE7221ThYHRnCfVFjdknuCDomj69wk4SsR+ZPXvSXs1jSJWKhLfUSe9n4bmw7w anet2@secant.private.linuxbox.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC5CMht/crEocrHqGjf0sxE9AhNcW3XBxwlniJwaBJLhekLQEQJoxAy2xUUmPYb2/e7rdmTBxzQipUaAGWVk1FHANac4cbgXzNT+9cnF7EIRkPa/x9jaKNiU0ozMhu/GOoeDXMrHIDgVGWPF3Ag/r17ORpYnuFTjsqoCWyh+mqskV+lmAp04rv2tvqouHtEs6gLebKNtB8c6twhrBKF62Ie+rSr3r7hQaqiZitHbg/isbU5RMBl6H48wHAVSK2/uT 2019-11-25T09:08:24.933 INFO:teuthology.task.ansible.out:fMorbnuEYRG8HvlnPRvQHscJYKiZSt/G7XqZ3x/7c4czEF+OpdQbAgg46vh4fNVMxZk9KHwEkOE06ROuwgAejJ mbenjamin@duel", "key_options": null, "keyfile": "/home/mbenjamin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mbenjamin", "validate_certs": true} 2019-11-25T09:08:24.942 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.954 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.964 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:24.975 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:25.008 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-11-25T09:08:25.340 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHk8c8wUTTYolOOmMlniNrsjfRyF9pCCnaEQ2DXR73P0hbFMme5Md5cF3PTtPIOHSNkWk/qg7s6gZD8LGyFmloO42RBgLsW1qM8RWTDbPy5EQdrjxL9byVzBBCDoQPIS2M+0XnhYa6CBj38Sw8XPlS41CebAJEOQeyLG2I6CAI0rTij6gsqDfALbIezd/eWvmbksKYM1Lb4Gvj+k/OL/xDZzIZHF/P+mK+trY45gsUHxKZamkeHyAT4f3kiNJA+UwzTWGhafy6UE7yGqhZjt9xOZSQGuwroL7EtAzntxayPRNxQPfGDV+fYYf0QBbTtHyJeikPUuw7aR1Z3KwuSmWD dis@zambezi", "key_options": null, "keyfile": "/home/dis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dis", "validate_certs": true} 2019-11-25T09:08:25.372 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-11-25T09:08:25.374 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-11-25T09:08:25.687 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "key": "ssh-dss AAAAB3NzaC1kc3MAAACBANaS5gZUnIp5UL/PM3xOrQiNVxfona7sD1XEXGYvNUyoojPWWAvdQRwdA/Ay8JsAjKeWUt0IwfWaPfbEoXU1TKAI+yyv5WzQUZAKmBFuYM0CUQnKrk72Dp6n8P2tnYen0ect22xEiW2+Y0JZ3wfV3IB3BQPoVCC9FQ+P3LHhYii/AAAAFQCgSceiZFeNItwJ2rJeQgVVTQKljwAAAIEA0Gd2VNCbbQ48vAF+m/i/CeQHJ9Dom9IOBN3v8XnSlOhtNzY1pJ0NRzrFQURCCeVQJaywr0et5FCXIDVqTuR4qfCu5XXU1KHpn/AvHstOd93Y0XYMwSXfNszOtpH7w9SEm7uBfKnLQNF843/MHZL4ZIdfxk73YjfMjrYFcvJwvM0AAACBAI9Wi2wG9Rrdbrgk8tqbnKMxxJCYt/Vakn5mQSc/WHqMUn14+Kb7oBtfTsfPFNK952FzU09ptNQvNWTPNMniI8l8lQi2B+Z28XUlFLL76uxJXWko99xFpf/d1kFBL0M7GRGmvvrMfeTCI+WIqh2jBVqbl4o8kjNx4p0VWyoiWeFJ gregf@pudgy\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAwSe7szyqMTqKK3D/JsXIa1PtPf1p8Njk0sMDSsJPA0gBGysmSgstke9fglOgeNIbNOBNDOY3vltuNCgUiQgVWOYhUaVpj+ 2019-11-25T09:08:25.689 INFO:teuthology.task.ansible.out:2vtSoAWpXMQbNaWPGKhbxvJWMCLVckLENyoUZUBKLmTvRyKDjjq8CJ1TnfA2Xfo3a796EIOtD6essU3lAVIfIUILGD+HqptiHSB0spUJUVRxxlO2xw9rLYij1eUPRHrr9xWvDamr5JtKVtCLOL3M9QlltHieKlbJSCZq/EVxW9NAGA6IiqOhw2E1k59oGZUq4Qg+YLgYkB80Ms7nb1xpjQ8txvY+73xL/8i7Nqmr7ekKJfyTpRe03nCQ== gfarnum@mozart.thompsonian.net\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzqJQfdAKaggPDgSpk8P6QjLeyQYOz183fVSKnqAa5EeaCJozS1dn8z+iphswT3YL6wqIMqVHMfkSUcEeTBwZmvydaghOMJEIQSP1PCzH0V2XprELrEcBC3QTvaElPjjEYgBVUvEw8sL/RIQ7mgt9vxD0NcCDspsz35fmU222KzFxD41PZ5XzHQ3Lumwdax7mcpqrw8Aa5/tgcEqAIu54HYRf1Qmk6+ueMAvpwkX3eiSEtkVvuV1Kxpc3/0mOfJ7cn+eyPX3qUbo4oY76hIYseoUIwEaVE/pNW+pjwDSy4gSOcuxYDEGKzCwZPZeQtRQWzRt3YNafxi5xsByobGLlZw== gfarnum@GF-Macbook.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1ik3LbyKsPiGwdh+vGrKSYLvhMg7OkqMwDNYU5LqWBSotogxyGQJUuD80ZavxmmdXC4qScddHtWOJjtyX1CTSDUe7INpSgGPGnmSExVFpcYRnmxHAB+9M/c9VorUkQ8IOKLmRO/BdYma3Rl9NzhHI73yd4zJlx1K4NuQnhnj0Z4yXO+k44RkBWDdMicBB5lBP0hETaH2Iq1u5Dai3u6N5N9bSPXv2PGKvvU395K0Wts2CxMhOBc7LAixysNzNWLrnG1ghS4jPVG2TIHOElP5lIXUv1Wlkool5rTUWgnVY/7loWkDus2JryDM7hLGFXqgT7HHcpI4gSITqRhWxzsUF gregf@fedoragreg.localdomain", "key_options": null, "keyfile": "/home/gregf/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gregf", "validate_certs": true} 2019-11-25T09:08:25.720 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-11-25T09:08:26.038 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDgS5EfJyOxBkl/TtJikfb38rm6muQo7MwYufkLSBw9W2jFzt/ZMbVXyVsdZM4I6hWfZm1/q60UcER4yoA1xwYGOTmbFXrLUqqKxKTnCdyejuQ3miB7aiuisWO4vjb2+ppBwL9Mx775DPeQJehFyFK0qgh33tZO/9LWWYy7NOZdA9L2ei1PgMWa0VBeijcDR5GQSXzh0Yi+h/Ezu8jevu+NkW2IUJF6zPGjd9M9lAy7KG2dQjqqCX4XWmUS+MHJS6f9bsC5i39GWy29FvX/qFEArOMzuYRIFWE8iza95KSBxptb+Hw8PFBjQ1kTh6nC5ZV2X3ETcoj8fToTcgW3dTRF jdurgin@new-angeles", "key_options": null, "keyfile": "/home/joshd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joshd", "validate_certs": true} 2019-11-25T09:08:26.049 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-11-25T09:08:26.366 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-11-25T09:08:26.403 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCyOWnAdvzh5zR2ts9e8g0wG4wZ0uNr0YJUCncAOTJB0L8xkQU76jWzX3BRNU5Wk8VtLvpJAQ61P7UXyzgGWSO9FHZjD3RDrc2V9uh/lqRKmx1COtDvcart/JwzwHlsWdrYdcXvEaAiFrZJ6x1jJAjIL6tvMO7bqBz4YNruKpGWsQ6p5P/M5iIEPMj1oKanLoULXqReC6eYFTw3LNAXjberPY5GxYwaKS0HuzySRvSblShyA85jawTSlEbsOSnMqqGu5vGJiC415UFQQ5qKBtzrgAjlUgMbaqGTnTmBYn7L751gIWRetgWqwuHgfIe2ZGM3mmh0DX8E03DUbEt4UxyR dzafman@TrustyTahr\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDDtSwGzH7V5jzppggL0+nYlBSM1HqGjkInwxutra5iEyFoBuV73MNSBk3HrvIVGN13+PCdHf4Oy8ywbhC/4yKbxS1NG76d/GU8ijZFuBlEwxwHZpcThpSTuYrNx7FJVp8Yjl45DtD5s8FMI/0YZZhLlIqT6QCTxlTIPXDb6zefEWvAVziE0bmhRpe4B6AOLOGTB9fivQr5YzxhfaSS5Vu4U1x5gmosouSJ+aMFrQWLqa/eoCy8t9PnxdMjlHjHCZ7Qv1JfpaAPw3qOK30GrVeXR/hPN3aFodTuIJpzdynjk5HO6/VKW5+mWBO7pmQqn9LNQEjADXbLh7PG//nOONbH dzafman@ubuntu16", "key_options": null, "keyfile": "/home/davidz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "davidz", "validate_certs": true} 2019-11-25T09:08:26.695 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvWnKfzTd574C4GXg1mr4pbJOInKv4qMRf0DwksTzqtXStlhT+PaO2C8p48U2Xjz5hmPK9J+iHA2xKwI6lKI194wUo3tW283euRyTCN7LlbV6tgD9k78hDaeA+53MHYmwbxjHEqAadqrLU+68LfmL7gaaUSToP/Ap+hdcDC7H9lulMcxkUeahEiDfG2s1mD8IUM6XVuqZM9WEARKpF1oSEBKhMMob9YJZPpJWipuAGkPvDA0yHK1lB58Yr2x0yaezVjQc7V2dwaY1gXR6AscU5WzlwWNNwXHqPdyyWRaGx/9Fk00yfzX4V7ZqTvKdMLKwB7EnamduTMPsaoKBUBJCh gmeno@gmeno-virtual-machine", "key_options": null, "keyfile": "/home/gmeno/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gmeno", "validate_certs": true} 2019-11-25T09:08:26.706 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-11-25T09:08:26.989 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDkIZyWnJXYv/z3EVeKHatjFUAdaomIMqzvFLrA9VskeM0Ctv8a83Wk3BbYvTFQWALjZjS2HVYzi5p6kTQ4qxXlcO7HxRw0j4otNauilvgIc4Lu5N4pUtYf0piEo+nrXz+WhELfmJ8t1Yx00uGEl9pOF04SOuXtpwwrCFAO6WMCcjDuYhoI2VBKO3W/OWcIuQ8ZY3j4iDdofFOUGyyoYVyOi/HILKjluYB+ues5+wBEkDuoi3MafCnIj8M0O5fU8/JoOTksOu779UN6blPiNdEgnwD00jA+RWYY6b12ZVkiF4rUVGCiiKKUwOOWkRhkA5OEcBgZ8cGWFgGfZM1GJA// ivancich@sleepy", "key_options": null, "keyfile": "/home/ivancich/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ivancich", "validate_certs": true} 2019-11-25T09:08:27.067 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-11-25T09:08:27.343 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsTZI+b02boZv8jVLHQoFlloD2WF/3xT+ZfZxzwddL7Q6eqo7j8i21LAbLAMWXYMyhiClrloDeHfCZ42gG5bqSThF8LoCw+GspF6SahTI0lmptGu69/OeHqqCpW1DnsWB3wWqfi90XbQJrpg3cdRaqOCmkGVJrqfFWzff8Zg9IuojIH/aO9drCTRGa1pHHCyEdW9ZNeYFwrMjfP+wJi/mC/ABAoXyDy0NWT0uIA+36abNg2/ptok90rdan0r72hjFEz/1hpupKWri+VXAepiyDMuSgJm9uJCXDFaNTn3jocdX+sbVtltThQ2OBxe1T60vRLbXx3lvhxxZ1fUkcP/FF wusui@ubuntu\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD2UzyfBk1nZ8SLCtW9UjorMeOo4u9AM/YL8ZI74GdprR+odERZR17/QfwT5vmLVNa51+CQt1HcGodlQYfR/8n/OUl/gJyVagBuU2zgmLPO3tOor5RBfms5MZ2VZ9Vm3+WrCO/vrAq/5HVyGbQsKiQHXkIRSL2NIgfKRTOVTNmuTSXtZNDgdWGIBmcOFQv+Im9CRopoI8ZtJLQj7xfxtGtUas7FbWTcf0o92+xvWuGWq5WQgmZ8MUIfvBLYLmicuVfsWjZ1YsC1PNhuhkwrW7MhaD71Ej3ll243tG1QD7eAmo+psIp8phlU5s/5FeA5y5OzDZ2T963oHGhq018TYO0b wusui@teuthology", "key_options": null, "keyfile": "/home/wusui/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "wusui", "validate_certs": true} 2019-11-25T09:08:27.416 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-11-25T09:08:27.684 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCtmxLK2rSomB14d9/fkAshmsXQfd6cJ+vEb0MZDaUT+OLmr1vAq0+H5g9w3hApJ2UEOFSaiz6ywgBAFeBeu/KmUALOhJIKKk0imoTunYclqxVPBK6tG/2b+ME8VE1afoEpMem3Bzfkrq0pDt6DLFBDlt9rRIqtO/ZlqEayWkEwTXJ7pwj+oJS03Z3Y7q0Iqw8PToKqJWwy2OvW1I9Pi0Q8s4YV9I34rSO7Wv4w43AXpFHRlvJWIcI5g5WFI5tnjel+kUAWMZft9ynFoAP5iXdByq897r4jAF74hapaxLMsHBz4MyYmPZqS76Cx2e3kGqXeetPinfcZxC3ViboR7o/1 root@zhyan-desk1", "key_options": null, "keyfile": "/home/zyan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zyan", "validate_certs": true} 2019-11-25T09:08:27.766 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-11-25T09:08:27.767 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-11-25T09:08:27.994 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCw3MhMNDYjkiQADoC1mg9ga0Yj174FHIFt4RQIC8qK+6cB0N3CM5R4JwP8pR4tYkVFWmnYRScYK5VVdC873Ac2BzYxmse97zLJTbUnYWTdY+OFi1FUozsy3PwtiNHSd48Z/PBvkinQIey2QlMKaW3/yz9Of/Yfm4prxg9lBsd6/tOOFj3AsqIQ1OVhMfgeVEkdCL7AAbEYyMxePZ3UFrDwj05HDLgmFzx1xsNJFOuBwLfU/QNHoXJQrcVy0rZq9fB5rg74iz1JZs0uDWocQQK4Nd3Rv+wRGBuew2fmkrS2bQWXzMoSL+/i7pbf74LKcbWAE0wXEImqQeREHxq4ZZ5J yuriweinstein@Yuris-MacBook-Pro.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJUmq7btpBMjjODHXp1888xoBuX+rhiZg3ufuo69RgDxUMymJn3CPJ/5NGXoZ3SGAWKk0w0Dx6K/VO4y92mAV+cO81y8wZR4J3fPV3IbWRjN0X3IFoAvJi5n9v5hpG7z3RZCGmQnOZ99m1i9dge/3S8NpuPLIvYkSo9z00d+mNP6D8rGnqOmEBfEhUQXLeH7rbPwaMWPgB/hrpiPCyLR/+sGC5bq9PAsKnocqZ12cnvCtYxpnA7et6nL15ftEpUrKYAmRDxotuoUcsllP66GSszjbrgN6OBNDSqpGt/+RSJJwzZEGQ073gJuMveKy9tvD/PyEttjvVKL2KsBuXGSuL ubuntu@yw\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmXWgRRes8bzekTH2fTcxdAAquFn1SjPhws1 2019-11-25T09:08:27.995 INFO:teuthology.task.ansible.out:Z9CkxNaqKe25XnzPehqrhg9etouUwvORQHdDJdTlmhTSYtn588Omq+1bvniab0KzAYgGZsWdPma/3E9qF7ChXIMik9Umip0drzYTaexCuC/Nu46brTxlnL0vaxnnVjDwmHC0CEJs4qN4e2wSlsA0uBDRWjxWFMdTUgSzII5xasHypUYZ0y3MNdbfY+SVF69DEDH8EOQMGYm8JQXNJFloQvmvLUFqT/63FsdpzRVTph4A8hvI1f8i6ULb7fPQN5ZS62wdXd/KvrBXrCvhOi7R63RM60mkMRXm1XajTGmCKiikfiqgTr yuriw@Yuris-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC41kRr6B/RC32DrQKRByalH832H6BpCQF9XdTvgYi4+LJt+kAuIGLDF4xm4uApTmWRrLspWm0Kksc0ger03uNx5qJqMEUnJvjxJnSDC0/XyxWapVxIviFkvv0BjbhVNOOaMU55ywrZ9hHgMdaFwuMuCOaB7f9Tm6IUXm2k82OXueeIG23Kziq+IQBysWOjvNUy/LLpl7l4+xOkhNUupUiQCqHvFH98VUdYWkHeyamafGVZmYbFLrOXgmviKo0u8LPxcT7psKLk0yFG7fG0DSak6DDdKJqb+G/NqMSPxRSAI4FbXQ6UOZb32Ozr1Sql9yfOXEYWZuatn7W5F8DjSv3p yuriw@yuriw-RH", "key_options": null, "keyfile": "/home/yuriw/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuriw", "validate_certs": true} 2019-11-25T09:08:28.098 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-11-25T09:08:28.099 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-11-25T09:08:28.338 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxrS30JmTZbnBd73s9PGZYxCkWH3u4En9Q0bYJan45W+j+y5vMzaNUVsp8We1tphqPQ9B0ygi80OxbHWu5gyv4g+M/JFtrnxv3QgBCYWeBWfh652Vbdlk1thl6btFnHUhVMSn5tkdnFxyVDJHIROoVXkHRaCNAUUy7tB60IhPAMomNcCGu9R8+opONFLXKTIfkYQ1cGbysr90DMs8ASEkvxdv40hIWhK4ufvL7CFFtgXuoC0c1WedBFOkr5LDegzL13xN97K9o5J8CAU0MUKk6H8KNKbQuNq1zANwymMFY599GIlxwai8iluJgfOObDRdCDmKM2f9RAEUXQM7oYoH7 tamil@tamil-VirtualBox\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvwiWVhNLeOLg72cpVIwbUYnmFh+Cqf/mE7Rc/Xci91on3GPrq++dC8t2GhTB5Zyq46EE27SmUkiZpHRnz7xwiJKM/tTgNEkffXDK3j0pA2E22iqV9FpfSA9Zdi1Puq7CPT7/rix4rF3SPvMtg/R1fkyornLFdzhJiMExCKJXGwf+QsKk4NVnr+nO3EEkKOoP1wio1bceZ53rCt9Tyu8NawAB2nL2fvtBPQDAriQ8bLX69XLbOfhFI6GJo815/j6CQxNmvA0l5ABJc8+A71jZmndzumFRgqBuhiCM5ohsGR0JGahUQjFEA4quNTx8EWTGRsqhAYszzpqJzuN+Xb6tJ tamil@macbookair\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEPzEyDGI7poyXVPG70vgQpvlU/lTt6Vvv5CoHWJoSrHklDdJEqekhCB87BX4DTrqbb5xoM7yfxle/z9UzgVwSUWqsPdZHA4EbrNFRKFxi/gc2YrygVSjumTyzf9WJJ5vCZvTQexOj7rjWVnJkjXHKMXZXIAQN/bJf6J21t+4A3XLrl30/4XXyq664NONUNjBHxHtlbS2LM6Th/hK4B23oj2BugX2j/50vWgd9H36bPr8+JGskOi6MH2ndZv5pYtLznFzQmsbixjaN8kCuyQdlRCOndPEFoHFhDla2MeB+h3XQLDXW7J1TR2URFSfcNq5ndDS4i0b+t75Fjhb7AtLT tamil@ubuntu-delldesktop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6tJ9y3solDCzFdXcczO8GJEwMqMYXWU5kuMz0rr/US6F72j0spSNmwGemMOZE9s7KtHgior/Ufw8/vMRkR 2019-11-25T09:08:28.340 INFO:teuthology.task.ansible.out:hoBYB0C5KT9utGNqrtDdvH98h43XnKZ3K/yaHv/T+u2EYWNgHtOXMtgIVFnGwSD7RzF6TbwSfurprTV6ZUyyOxRcmHnkmwpbNwiwIoXB03k2PITdYCC3lEBEY0OC+vAW/E3+bFbV3RSEA0Cm31jmMyZyPeYiXiEnPXUIClsoDj34NwJTIgctwIsqM4eYUBg3pHliWYKEWDs55HZYoj3W/6Jw7MASzyAYFSmP8rQu8iS7tO1iEBRGOXFNxHirGQ1WqUn tmuthamizhan@MacBook-Air.local", "key_options": null, "keyfile": "/home/tamil/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tamil", "validate_certs": true} 2019-11-25T09:08:28.425 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-11-25T09:08:28.674 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDYGVqKZgLx2LJ49x0kGo/Z5r8MVefSUCYhhrK/DK8ht+YK1AqLXErjAedgpVz0WuwirRjh5YkiXGMYiOdkMUYWGk5nwCHEKnnCpne7Xh1OOqge/QGHlUigWVH8VIHBsZ+wTW25mqOrgDSGB8+Qk7Dr2o1sLGUHcx7vITDWOrer4KjF2+H5TQH/7irUVORu6fzeGKtGyOrTwQ+ZT1+fuhfznDULFGyvKGdcXxxNOg+21Fssk2nCNTlgRCGjHSrfelfuMT7QEt+9puWd7MWXYQg7xx8jkJGMpEQRFxJTxnfUlUXf3x2lc9Q/SvKfpVMtc0N+bk2F9+euyvRvjtn4LJNx jowilkin@jowilkin.wilkins", "key_options": null, "keyfile": "/home/jowilkin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jowilkin", "validate_certs": true} 2019-11-25T09:08:28.749 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-11-25T09:08:29.022 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDP/yVttBEs7mpVfJL+reXb/BfB7DsLrA5ArVqEJjySi3NOz3e/wE8KRWbHD1UdraAU3P8b96GdAQxGOs4n0muOMejmaXWZCP3HkRL+rcd+03ZI1lb6ga9XjBARZfb2h+kg5jTJp/Ll6mwAYbSXvQ4MuDQzwGRDX5Q5Xme7OZ4Wb+vBmmkihHydySyP9YXbWWsBw4jvZCFSKMGsUVy/HIt1coZx/PyCG/K2Kl0nrE5ux2QqfdTIpzYo4c3y6uMIHDvt29lRmk3Gnh2//U5eN8mmRRKJlkvE8DKy3VYsl0JsP9pTWUZEYgV0EX251WTO0akQ2bKuszqd9Ai04RWnXUu5 brad@dhcp-1-165.bne.redhat.com", "key_options": null, "keyfile": "/home/bhubbard/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bhubbard", "validate_certs": true} 2019-11-25T09:08:29.086 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-11-25T09:08:29.088 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} 2019-11-25T09:08:29.398 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAyRDKb+InDsnW4E/S12Pr6rK3wDfUG3j+iw/NsCWwN5xpJEA1/NhVlRzY8/jejc6ddmnNKWbm/UcYVrbo7yClRTGu45g0Qm9lmfvgDgi1k8mwUCy+edbQ/R8FB9cIHHCnUuWzRmE2XGaeDiSVQgiYqq3mOPY/hwe3G7jwFalYPsbmY1jNqpzODmKFMRd1bOwdHEOMTfh0owHWqhhXakU8em30lYraQVYH24FplH2NUBArxmTuctyfweHxx7yH3ZFcXFUpUMSLuVDWb1A7fMiL6eW6wiwh+dD+q5iFNvuioZITP2EjIsb9nS+eM9bOJZuZ+5pln2EsHee27LjunbwgWw== yehudasa@fatty\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDF3PxJYgI3tS7SG+RnUvxMxr0/XwBAio0oiP4AC+o8rUYYyPd/RU27TR1kphbuJhQWVtaZ6kFSBQeNQUxv6rb8hmMzvlP1FDEoGkjaOHOuYEc2DFkkEAfaR2AlNKn3pUQgLE00TLHctCgd57Cn2Ppt8xEi/vTuo0gX3xjWOL4N9QDElCXKNdxdyyJ/RNb/Mid+YxgKKjf8ppmhYsQ+Qqu4YwPzLkfK1Qb3YtR6Hbjb4jxC7C1Z13h0vUgHe7qSmUCYNo9eC+lEgNmmnLb2sXJaUUJM8MEex6v+4muWM9hV0cq/Wplrx8fhMWMa978sxH8yVkE3Svqc/K4ZWN59l3xT yehudasa@yehudasa-XPS-L401X\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDvplQk8xkSXF03uSJh3H4zyZHRMjvgUJBgGN27Kb7gAlIQv306wPKIprOwp+jDhEcPnEWBMRJMUJuBi0vEiqmDHt8mg8ZYcs+Gi7MbUgBokILdJuSG89KAgq7gBCuIhYbSg35pfvXvIZswiXvySqMrJNB495GYj/FrcWEBfQMuedMSkIfNzqPV9yR5YRh8fnSs3fjCzV96vzu7O4AzgwICbM6XmqVLXPax/32Hq0dz7sAawlQFCJYPHzVzjwm 2019-11-25T09:08:29.399 INFO:teuthology.task.ansible.out:ih46TRJnXrxewsRhk9GwKfI4SWaFIRdOdadGFSLgdZ+2iGE7hTZ3sRgkLOj+4zKa+xTbtjjXfw5up7J3hAmrK15Tz yehuda@yehuda-940X3G\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfNxvAw6cb33jg36vw8kJu48/BttmFJ4BFeanYqwRgtWZrKpAAGHEgCiNeyIttMrI3t7Spcdyp6dy6n0P3WUFVgHWEYtGqXwXqDC9UXISkUJVVDZoEynpGC92RghhzH53+7jX1Ytpr30r/xb1ktryTPCoEJRqyrg2j1b167i6PV5tuJCA+77gW5LWBmMXJCprjnfnJ92Gp8ZewBB6gL1x+3RFkMxGsibL0Qw1CTKfyAHNYc/jos6YiL0b42Ha+NmKwvzZ1GgIleRMleisSOyKPUlPdR7UnmtEXa9bl3Gc/hc7Zt+v4iUypr2WL0X4Yl7rJNH/AAd/Z6jPkAX9nHYzh yehudasa@yehudasa-desktop", "key_options": null, "keyfile": "/home/yehudasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yehudasa", "validate_certs": true} ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-11-25T09:08:29.711 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-11-25T09:08:29.733 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIDx37xbP0jYqopygG66LbSMRPoqpGUPzQZ3G8eqJCk2I dang@revan.aa.redhat.com-2015-07-17", "key_options": null, "keyfile": "/home/dang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dang", "validate_certs": true} 2019-11-25T09:08:30.035 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-11-25T09:08:30.069 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2YyDoeKdaCuJif79AxJAorqpUj33WHbHu/eIWpHju+iIF3f94K0qyEXgs4AwZin9pzfyd8N1aKNT8ZFPss6L0Xk68aO/6tdInq+1lTMMUC5v/LBqsIV2btPTFHrQWA0R10v05aZFvFaSqhHNsZaj98vQV7P+jiSs3yPWqCIWUTQ1VXnCeQnB4R5tob0gpMuonhkKq7vrJPhf6ge9Av+yjL7+8oP5LJqFJ5npDxkwZ0u3L0QQ9ftAT/e7Y7ESpwpgaI3Trl0EwAXFmiLjXEUA+FBbE7ko9kLqXqRjc1C0BdRG9sO0QKBGDBEx1W3w8rCPKgaDlVyqsBT1yvIjqqXDOQ== borix@dhcp-31-190.brq.redhat.com", "key_options": null, "keyfile": "/home/branto/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "branto", "validate_certs": true} 2019-11-25T09:08:30.369 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-11-25T09:08:30.412 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDO00mus2XPX/EfUZ/YsKI84TnWj5TbZP7Fs01EkoOTPgNoYysVdf0PwhAooz1qPrv+HUsUMpQT6uNYfCHyqx9rScR+XUlyQuFe7Sjp/dn3y+b1PKa6G4HX7f1eejR3ZaLZXA4R/8oucAXqQS5pDr3Y7prwkrp9cyk8uawBXoDcAlA1SMIPZPrtO1+iZw20NDhixsG7NE5jMFfw3q8xTckZEqBbteybci7X1v4lEYXCVlbSbUhyZ2Af8HEj09JBDOL2bI3yLvj9a5Oj/HtKubyApzZpA7rgeoii2UZQGuSRI1VJktp9YC0zGk5S843xYW92utbALKyQ6DdB2F5gc0DX xiaoxchen@LM-SHC-16501223", "key_options": null, "keyfile": "/home/xiaoxichen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xiaoxichen", "validate_certs": true} 2019-11-25T09:08:30.718 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-11-25T09:08:30.746 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCz7e2G3Ziaus2yMviKh9s65e+yaqy1+ezs5IC6US90/8WumNAWqC4Yr3ckvCc120r3VwpJWcvaAWCF+4IZeO43lSNLadbCnJONed5YklcUuIJvjC1eDbZTMLJ7h0ijgNXgAFmFq0AV2NHuE2L2VKXk2hsIqDVGjZLUpi7W+ic07xnScEoXXDSk5mhJ6bKmFsUYCC91g2roYs9Ui5fGd+ey28SFOADI7osDLpLkiTF2OzGel93o5qobNZwKR0AxGsBIta17mvf3I3307bGnvugDXpNB0FaS1QZhIzMO4G4LZL9zahCvv4wm5GmuO4F0+Ac7dwFu3KZKtHQe3g6WHuAv ffilz@redhat.com", "key_options": null, "keyfile": "/home/ffilz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ffilz", "validate_certs": true} 2019-11-25T09:08:31.060 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-11-25T09:08:31.062 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-11-25T09:08:31.108 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpGZK2cWUtE85Bt7fl+kSeiC0lM/EN+H2X3ePGq3LiD4g5R5n6nJsLaJaIZXdJ96G0JSnj07G8Xu1CkVbGDHuVg2O9tnxLhz6tzDwFYNFrQIpALlQwOq8OiwGDDyFkMXQ3JqCCWvyD0HLysOJQIBIlb/o92ZCSAGOyNX1JjrhXF+mlBk1AUW11UeZOeGZQ34MKHapH7T/DuexCz5gfZI/jY9Rg3hr4jb1zGttvAF2eeLJCjqD/Vi4mowYB/Kn/PJhNixeA2sli2i8r6yO/bFmA/PWnfSl+EqD4mUJWb5ATeJ3hyW8qVLOogY6XVu4Sd1VgL7l1RYzXuxJV3Rb1Yw2T joao@sepia\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDV0jbKpbmXW1bhRIOBIdVV1L73qjls7s5pxiW2EMPvZLYzxLBE92e3Vf8bSMWlm1Lyb7a+u5BR85+2c+h2F59sgU1Saek5ebPpFU5qBn8AzYH18iMVSPciByOzwS0NXwWnksckWxUkQgP9bZP/1ORQ7jWZEOHZ5mEQ0oMkP0bfG1cpFrNTxOhDmwbCiQ8dp6KeZaMg4hErALr5Eh/EESxKIkuwQzk9bDbtjPRDu0F2dE0NCjs0k0BXmDA80GksxuOacRxE95K6/er11qst/VqA7o6BfhxJMTXWMdj8NhzIUNjZ8LqLTG6P/XGR24sA2d2WBhlEKRMZNqeEjzmi1REr joao@tagide", "key_options": null, "keyfile": "/home/joao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "joao", "validate_certs": true} 2019-11-25T09:08:31.352 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXvd7F0n4DeBROTLYDE4lItYn/2S1cMM1uAkkwissUh4LGsMAVWi5tUI0EUC7I2BUxACvIIA4yfkhe2ATAu95UQxKWVql67sfCW6WwY7lNa65kQC9S5dE/+d4vPxFoFSFnri981laJXOjbjTsWD7j1SGfqbRuAHQKL8y95DQ3y/B7/1oxTg1jjK8BTr3GAUp8/7lNlWE+x2S3gjHqqH92yWU7OBbPWRZjjkVkXS/5CbWDDUx8nbeGPcAIfM95rRQR0VkwzVi1FWsXd6qIbZ9TqOhYyLR0fCsnDuveNV68FB1ddvqCLdXRxPLmucwzdjGMbZ5EJ0H/CAAutj3Vjn6gh jj@newdream.net\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRXOkus/WHB3llEH1u4rWj0F/bZSslq4/o7pBUGFYrHKqa2iBiCj4DxBEsaReWUm0aDyesGuEVeoECTtzh7byJ6iZHFbf0McubxnuFMImneQ2mB0+raa3uWblwnyiwAqiZgtcJGQpCxCnrmLzyNN6xUUmzN27cDOMSDpbWr3VdPSBH+ixIuZI8r8UkSbmlX30V2bvk4lGl2SPV385tAvRPRU9KfMG9AgwNv+R2WCiA2GOrjes2VHm7LneqI/vE5yOxJsm4tPxEkOpCp3/0xt9L9cmPz47HnzJ2Qure3kZmOsCxOX1+9dXFIwhPaNconD4KDlv4MNs/M425Gp6KKnnV jj@laptop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBjrQ3drodiPr83uvpUUf6SXZ0utOFIfy5Cj3BGcn7bY29BKXCwomAUmmLMk76cGBZ7jHf3ZJYid1ESbHmvmSgKFShr9JFXBHvlMdnyewi7hFix+sjX9ZtuZfSaIsBbU5Jlz3RqHtDrScS1wJRh+ 2019-11-25T09:08:31.367 INFO:teuthology.task.ansible.out:oDlH6is5QWSCABVWhUk+hJtQFg4dfespqomt88NmPZuR0aR0b22hkrMYINXP0y3iz3Jjs5vg8td7VJ2hXb5yIBk+Izuq1LnjPyRCZv5LkZMuwofexpfflLKeVHw0t31HXuzBUZMoKqXiGJweyPDzY/xbeTVnYvQQEvoGxRDpaYuiGSfhFW0UHE3HSeuYfklO2x juanjose@aurora", "key_options": null, "keyfile": "/home/jj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jj", "validate_certs": true} 2019-11-25T09:08:31.484 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHhYb/mGgHbEntAN2iPFTV1k8MYDs+jtsJDPE9MWtUZXksdeG8cEfXDBGhiBsprjELsW4+czJBqqESugFMevEysTVKIo5lc95bf1nApxfiRyMnAxb5wQ2OeLY62x26TWJhPu6HMajV+gb8CKjxDobf1Ix4ELbYlL77SOH1VVDtVZ6HiwtYngXAPMONCdI99cuwJlBpv5RjDoLwljOoxEXIh+1Fsvth+iymp0rIPuJx91DzYYgWZFfCLPHRgDy0H7YRdQWUzhrxTyzuT27vsNh5L8HgdowzOu9f2HwGzlTlxli94IJLMBY2359xsqjSswZzAv+BiPDAZMUwJpYvRvSt nhm@clusterfaq.org\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1JAKymiRDyVar5P75cMycxz9u8VsLuNtf/0t1na4ofa4YmanKswPQQz1Am82pvyPqXlBbdIGVneyMrkEoFAk4nTtfmCl7WdtaxnH87Dr9BOLEp+lIPHy7jpS/dcVmKRPB1qCYZ5GNAZ9g+DrXdaeOJdUOpIK49BP1IK8Uw5eBWAx/ae6/12yz+d/6e9kBvO7OUamMSne6Ghe2pRoI8jFOdWMbQy3ofb5wSHZGPNDn0YTGbNWHlfEAcrFmGQSTwohca4SqCJ1NgKnYu4qPVmb3Gbpm+MpUPs8Gy++rQ8wVZ5xWZW/yReIbJCqqVY4vrQzVywQXRucIrB1VyvShuNW3 nhm@metropolis\nssh-dss AAAAB3NzaC1kc3MAAACBAM7BfQAinQmLlkTgt7IIg3nQyyaJRiWA4agfu/HncrcAmepGqPCwy451g1eLMD3AC4QptHnV80Y47YQTyX1VQixIQ6M9zHG6Bo7IXzyqFHFpqnDZN25DaMM4na5bf1CwC9RtR7VWubojatPylOpflkChUBuAGWnzwkz2cxl4Dc47AAAAFQD35l/Lg4eTqZ83eTgJ5lihzHS/awAAAIB/ZdMSE6M0zEvdNPsWY28bvQ0QXN7w1ncVXGachA1JOEFGYvsbHeoywLpSYoiPLV57/wZT2dUzG70u80IQGildZ1WgQNn7C9r2Z3ZUgKmRGyC+PpYEa6nv+U4CHdMViBRXxNrP0bj2 2019-11-25T09:08:31.485 INFO:teuthology.task.ansible.out:CfQvUxUxrHWtOu/da2W23cpSs6Wz8gtz2QAAAIEAvf4vx/32Sy+kbvqOCYz0wu0ItlECN0qOk3gAMvphYMI2sW2ez2+Tsua4eoPRXT10zFxiRTkHQd5FSIvNWXV2iqed6rpwTu5K8vJz2duMIyI+xWLHo4XudtZdNxggkf+RW9JvPOMLMp4sCzBXxpreHBsb1CynndKeYVrgMLybZ28= mark@Marks-MacBook-Air.local\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCfDOM2v4h6Dk4pMWPQxxHI5rhFtLjkme/4hAp6ki/kJ1XOSzBQdQmD2UPDKvfO9lKO7v12G2Wt+re+0giAcSkfej4+xqV53u7d2pTbMI5U3zohZt8AgESx22DAxj1wBZYuyGO2TRI7DRQMxC47E8EJgcmZBBnMKk1Rz8k71pTmg+CI2vjWx/2x2bup9Mqe4uMA8ubjg2qRlWEqPKIKcHZyrAPdIlpy2FSZ9SPa0mJpdl6aYcFNgonD8BO6A0u2RPO90KmEWrRWu69vRvbbTDiqE0CkJJlAlu3jeFVD8US9vymBHSI0BDFUbukuyG1tAujytqGBRLzxQDU9JOYMP45v nhm@mocha", "key_options": null, "keyfile": "/home/nhm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nhm", "validate_certs": true} 2019-11-25T09:08:31.701 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-11-25T09:08:31.838 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXvd7F0n4DeBROTLYDE4lItYn/2S1cMM1uAkkwissUh4LGsMAVWi5tUI0EUC7I2BUxACvIIA4yfkhe2ATAu95UQxKWVql67sfCW6WwY7lNa65kQC9S5dE/+d4vPxFoFSFnri981laJXOjbjTsWD7j1SGfqbRuAHQKL8y95DQ3y/B7/1oxTg1jjK8BTr3GAUp8/7lNlWE+x2S3gjHqqH92yWU7OBbPWRZjjkVkXS/5CbWDDUx8nbeGPcAIfM95rRQR0VkwzVi1FWsXd6qIbZ9TqOhYyLR0fCsnDuveNV68FB1ddvqCLdXRxPLmucwzdjGMbZ5EJ0H/CAAutj3Vjn6gh jj@newdream.net\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRXOkus/WHB3llEH1u4rWj0F/bZSslq4/o7pBUGFYrHKqa2iBiCj4DxBEsaReWUm0aDyesGuEVeoECTtzh7byJ6iZHFbf0McubxnuFMImneQ2mB0+raa3uWblwnyiwAqiZgtcJGQpCxCnrmLzyNN6xUUmzN27cDOMSDpbWr3VdPSBH+ixIuZI8r8UkSbmlX30V2bvk4lGl2SPV385tAvRPRU9KfMG9AgwNv+R2WCiA2GOrjes2VHm7LneqI/vE5yOxJsm4tPxEkOpCp3/0xt9L9cmPz47HnzJ2Qure3kZmOsCxOX1+9dXFIwhPaNconD4KDlv4MNs/M425Gp6KKnnV jj@laptop\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDBjrQ3drodiPr83uvpUUf6SXZ0utOFIfy5Cj3BGcn7bY29BKXCwomAUmmLMk76cGBZ7jHf3ZJYid1ESbHmvmSgKFShr9JFXBHvlMdnyewi7hFix+sjX9ZtuZfSaIsBbU5Jlz3RqHtDrScS1wJRh+ 2019-11-25T09:08:31.840 INFO:teuthology.task.ansible.out:oDlH6is5QWSCABVWhUk+hJtQFg4dfespqomt88NmPZuR0aR0b22hkrMYINXP0y3iz3Jjs5vg8td7VJ2hXb5yIBk+Izuq1LnjPyRCZv5LkZMuwofexpfflLKeVHw0t31HXuzBUZMoKqXiGJweyPDzY/xbeTVnYvQQEvoGxRDpaYuiGSfhFW0UHE3HSeuYfklO2x juanjose@aurora", "key_options": null, "keyfile": "/home/jj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jj", "validate_certs": true} 2019-11-25T09:08:32.028 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-11-25T09:08:32.176 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaetIvR7YgpCq142fBQtOYsJrOMW+Wxu1beNXPJ8qzT5MxD05sdKntVE0DPveUydaeI8/vHeNjXDKN8mt1eO7Lf/rDKHOW/Y/36bUfQAODru9sdNCEJfSPgq1y6+X1ISljvLBXOOl+nQXlx3n5OjThNQbmdthgCT2Fk4hEoK7gn4+ol6foTkTdUXIBam8LuPU6f1KIG6ae1oAVFWLeavAbvmxUTmVl04+hKJWSPnvZxWupaPuiHvsImaQ7GwBjKhzL1wSMly0+pa2zbVYZKkLBA7Fx6fzP7jcEieOOPM1YU0jR5JF+WVzeRFJ1V9n75yOw6S4ef/X/zwWx42cdfx3N nwatkins@martini\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDRivdSX6cbPrdMrjz1oYgn0hb8VQo+QU7wD+AcCQi/CH06ny+2QjSr7bfdpgQ81Wgwqt9+e3KcLD5qdt3tneC8sgSAR5QUQrWKhPejrqdV+3990k8k0KoKIOZ9MGFc1FBv5IpuT/2d535iHJb0ZEg8mFKqKDw71sBEoQT4WElatf5DeVC507hP7GnrhZgtTPcrVddOLkmmqTQtLE1zvo9Y/eel8O2JMImMd6mdVBIcFQ1alU1wV4jLdMpagoPelvVTVeNkGRnOgvvQac5RXtGSMP1R5PoGz+1FgRhDCtdTXRfOdtrlJPtUjYiGPpIoKIUymlvQGjUrrK3WiQIIkIfDtFFu48EN4drjgQpCEQ2D0NMKEmk74mjx3CJDOF1Nj/2Fa1kjECiz4EM7sR8dDG5cW5Y8XsDoomgugM+rsOH9kWFFClHUJmdlNGWLhJnu9UQHLYZjQw3zT0F+FXUsUxKJUVTUdxQx5HBO9QFsSe13NPDnX64+SQ2VVStt8oQAvFO+6260zbux0RVaJM5dTNwnIHD/uI3PeR1z2b+iKhyj/9VP2uBRpGydLw/etQu/lS2K30S+kZIDAp0sqwXH89bdq3AJBipcryrnW1Ov9K5Mfl1UlV95/MivrnOFISWN1F9XR1kuIIYdCkI7hq/FHZ9gd84AjTiERfxRv8VCIHBVEQ== nwatkins@redhat.com", "key_options": null, "keyfile": "/home/nwatkins/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nwatkins", "validate_certs": true} 2019-11-25T09:08:32.355 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-11-25T09:08:32.521 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJD+2BHStkaMJ8RStIIz+yDJCGNT1wbwUnmuO71vNlFxzIyHGvCq3ktx2xTwZz/kj6ZayXYWIRH//BhK2/iK/v+u2WQMBURfMHbf0lDMmNfbHCDBjYvEYgFo4Jwvzq4WHus0AHh9TdVt+TpcxyGPhVbUJh8NZut+w4cRIKu2xGoZZGDeLgv9ijXvDPrCdBxtpvQnF99C0bCTdPM7MK1FSCQEaRmNUgzEj0H7R+gmWi0T4HUD8ZY7y0vWbYPqT3Z07My1u2BfRBYpXnv1655/b777TkSlzSasORhSTT0AkW25u90Rsj2JWr39+49/Ia9uOyFtZIirugEv2v7v/l0grN linuxkidd@zenbook", "key_options": null, "keyfile": "/home/mkidd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkidd", "validate_certs": true} 2019-11-25T09:08:32.677 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-11-25T09:08:32.874 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrRvNeBa+/GXxkSDmaim7U1wFpdBvgcK9/eP1I4fPS5xAWyxXI0s0ryX4Jwb7auhZdcqPjgUPQvIVVRridKJPz7l6jUlbIqdnZu4VHuaGG/DfIVxsSmw5aI5qI9Fg/s16JPwOUuzcLHsSSh8tFxY9L1yvs3ztQnKHW6nN5+zhPbW8UjmTNUksXXAeCM/OZn+IOOzzvfh0vVe/mghIW2JpOjHkZv4qwaWYcHg6+wWE4m8l84VIegDBiONknZXcQaRj82nHtfdnb2FSiV4+gKP18i90ai0iTKKXUkMJNFKp1T5MuF7UuJLA7l10FmgQJsZBS2Loa5oeqbHYeztNAfUxF JCLO@oh-mbp-jcl", "key_options": null, "keyfile": "/home/jlopez/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlopez", "validate_certs": true} 2019-11-25T09:08:32.957 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-11-25T09:08:33.216 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDFkUz8Wr2RPiY8+xf2hL4qMT+jA4mo1gmpIF8q7uVR03vX5NhzQVxVnXiyLCFv6susSMxpBafa7aLutqigXhDGJWUbsJII5yTnFYVer0QmudEsLmoDyvI1HCNk1IVY6aIESE50N3Ndx+agWasIU8TGKqJdDMPyiUtfrvsrGWi/91ZLX6aJSrFqBJtTpxDoNe4XL6QRPNv0wb8GF0az5graXkzVe3GkaQS9lyw8m2ougDvW1SLvoeE+lhUUGhJGkkCZKv1f+QNLhUHyQ3XkXqh4MrP3pPUKIuQ7nKiWDjCP6Db+ZcejjTRC19D2gLjhrw3Z2J7nn/loijlFzV5dp+BL haomaiwang@gmail.com", "key_options": null, "keyfile": "/home/haomaiwang/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "haomaiwang", "validate_certs": true} 2019-11-25T09:08:33.279 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-11-25T09:08:33.558 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA296in+8Sr9slF8hrzAV3dNz0q94Q4FlFaMYw08II5PQm6HOpabGOGOwOud/PPeh3lMUHhs45N5WnbgdfQdzz9lpDoNBcqxeACdMicdECMHtYWf2GuFUdTAnPBFVl8wFW5JcziQYJePUcAUJRXGGfdMLMppAi8VEuhMw/z3jz89UN+zf51Y9eT3rgAVSCR+Zn47kite4OB2S1aVMEcZBE0fPnh60i+rpN26s80fh3wG2N9JrPBEvst6LbL0aEixrlRUtoFHZuWNCIjntVMANa+idpuWSxN2dzfcoxpGLF1TFzqx0moznp0PnDs7ZAFFS1/EkeSCgBflyhPV6csG3VNw== jdillaman@ceph1.example.com", "key_options": null, "keyfile": "/home/jdillaman/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jdillaman", "validate_certs": true} 2019-11-25T09:08:33.601 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-11-25T09:08:33.913 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAuTvkaUtEaLbLcYA9Zw3MoHI5ZPudKf6CAbAZdhp4RMVujimR61TT6LdRF20g6F7pH7GrI/1WP2bu7a3sZyCZ1nCJCOhrTTS+2wwy7oGo/0yY4l+Bw6i7LGmJgqpO5dDxwADMlaHUwJ7Rf1Q+Pd7NmggkAVTmaN6qUGffz9hZsj7ibD/Z8/GFtQsCEMloTLtnz4NB6Gxe54UXEKz9pvjpXBhs0xWsJtDKaPFrbKRKswuH45XPuwJhGr249ZYtwQqcgXSWhOUcPTwf5Y6fj27aL2dx566DxQzqzNbcTcO96MguIy8Ufnc0kIOQQw9oHJZtg82zwM6+2mWr9J/K94Gv9w== kefu@save.node2", "key_options": null, "keyfile": "/home/kchai/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kchai", "validate_certs": true} 2019-11-25T09:08:33.914 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-11-25T09:08:34.236 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-11-25T09:08:34.259 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-11-25T09:08:34.570 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDAfEhx/xZpJV9ouKFW4QBC46cNicIQMzU8DSYl1GOP9+pnnD+AudVQubvrEePGA7TFkSar5Ccv4dnoQ2s1CaPAtbT+s+nQlf5OFO9P/qvhkM4JhlmH2XxgJAcOvZH2W1HXfGTc4WIuU/OtXM+yWMGlTXxqMqwCvGfqc+Yf8zkt0t3m0kBdWoFcZvdYqKuS15bvdVuiKz7GUxDrtw98yMXb/ob/E+Ygmmphn9dMatTeZ6SD1fbcrHh9oIVbYiY93+++jXzjEXrGzJh/gphgEtvIf8UPMF9HiXXPqwhxneRvghvZ/B05LGfcsqGAtyK09eF94hglyl9kJIdm/d7lmmM/ abhishek.varshney@VS-C02PJ2CSFVH5.local", "key_options": null, "keyfile": "/home/abhishekvrshny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhishekvrshny", "validate_certs": true} 2019-11-25T09:08:34.613 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC/r/Xf58szqJ3wLdTE+mRK1Z11f/+VABmx6804NO77ncdC5ZvD3Kx6WP/oje01iwWEVXNAgEOaXp6j+U10aPvBMaS30qf/IhUcrXwO1YcGzZ0pV6UZMxcTuJw6w1VXy/YtgW5BlIfSm78zabztES9BqCgjYYM2rml3L1teDF4WU1QiI97iW0R57yc0t/gqXGpxqSFQ+iNYAl0Q1MGUgYltLvPrhm9WI2WhR2/GivdVMEM5lViNlegxneAZRb9YZMCG2kmSP+9Zd2NR367OrlgzmgeY/9u+KGXQiyfaO330s+iksxVwwE3fDl+KnqXuHSJNccJV7iXzZKej2QAire4X dfuller@Douglass-MacBook-Pro.local", "key_options": null, "keyfile": "/home/dfuller/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "dfuller", "validate_certs": true} 2019-11-25T09:08:34.903 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-11-25T09:08:34.905 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-11-25T09:08:34.948 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDH/YT1mGZg50qDdngGQ4Ea+jfJGZ582J6pQbYkrvuGsyATMzNwk73sAbn7mgbpw7nkW/ol/vwahXHZ5heC056BFnqiZ7tjoDeotDx/1XxFtkbyIaVflUeD1vBww38v6IslLHcVhPilauPVZSfVzEHI7JB0a+9OwNrEH3+dt1B6vR+WRy2gHG/aEJffPMv4yv/0FgRWy3cEiUvGlLq0KVVTbsqqLQhw/u5k8bBIoibBtZLFAMTrjLYp9Dv8dxJROvmey6O2TuX6gCY1rLGTAArYULpXH73OLIqaksJKc3srgoJrTSZPO6bo3BQeLz4ODJ7uvTpvQIhacBpp/dq1ln7p owasserm@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCiqChxNdielMPBchs0u6bxWstqU9oeY2K9qf1PNwEIrwDYTrpDjsdoVnS750tOBjWY0wr9Gf9ATOJXi89151oao9GaUueQ1rTgmu7GtPlKsuBUVNMFzLEa0mqneqDA4AfknisFt/LnL8fLM4BddT25tmQ4oUGS7F17p4kXqwboK/DsJciCMWWc/5AT4NVwdnYFzIA6ta25/bB94kpktaZ6HDw7G8q/mK62Yza3EZfq8SP7p0sPL7oVsInQ9AutcVonmALgzPa9t1/MPGaXUIqARaWtwe4MSlNEIs4afLFyteBVeq+dgU01yeyjFivwVmuS+aPFTm6sjGGe8/gOsFDIJiVertOqi663m7q1WYGlVANfif7uL2+uJ9yrvOmrFh5s/CJMJBRG1SLfK1j/ivFwvPAQc/uMqEIRzrVZOsuNLPO4zHPWzmBAPUteb+2BqSdUFYMlBxb3lmhFFffroDqnOK1nbdAfmyKiosUizQgX2Gs2wjjtejuhS9dpYkNu+sBOujsxa2mZUouyWf4fXyWGVjthrCZaM+FyeSDpJrgwgRiSKNB8aRxB10ZNo9WZEy6eH2adp0Xf6v2KsKvryDndqBEkBI8zLbGrPVHGSEqiSBWc+NeAB06uSscJd0pHU3hDg0qW0EaxOXaial9qrBiwcFVKEr0g7LDURQ6KZsZy9w== owasserm@owasserm.redhat.com", "key_options": null, "keyfile": "/home/owasserm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "owasserm", "validate_certs": true} 2019-11-25T09:08:35.246 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-11-25T09:08:35.247 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-11-25T09:08:35.291 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDAfEhx/xZpJV9ouKFW4QBC46cNicIQMzU8DSYl1GOP9+pnnD+AudVQubvrEePGA7TFkSar5Ccv4dnoQ2s1CaPAtbT+s+nQlf5OFO9P/qvhkM4JhlmH2XxgJAcOvZH2W1HXfGTc4WIuU/OtXM+yWMGlTXxqMqwCvGfqc+Yf8zkt0t3m0kBdWoFcZvdYqKuS15bvdVuiKz7GUxDrtw98yMXb/ob/E+Ygmmphn9dMatTeZ6SD1fbcrHh9oIVbYiY93+++jXzjEXrGzJh/gphgEtvIf8UPMF9HiXXPqwhxneRvghvZ/B05LGfcsqGAtyK09eF94hglyl9kJIdm/d7lmmM/ abhishek.varshney@VS-C02PJ2CSFVH5.local", "key_options": null, "keyfile": "/home/abhishekvrshny/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhishekvrshny", "validate_certs": true} 2019-11-25T09:08:35.589 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-11-25T09:08:35.632 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDVFy1sjGvvHwS7Q6E61ykH4dUGfxW9+a7Xo3JMpTBT33/E8bTrYF6ZHSdH+uxScisV9Vll222NJCLXyx2s9EOcv4L88vWDTZGCaUb3iA5dweLm7EYi1NXT/p+jWDA7MAF1FQGSAQafTHpNRxLcrVPgsLDy35weV4xj7ShEU+w7EBDb75jlJXqy0luJLKNBrLzz/wDV9Oe6y3APwQBhtO7vGrVzrWEF/LWLe7U9ad3DQjvLM7+H+QVpNy+nt7us73K8vHDk1KrPlCCjrcN+fSGPD/ILlxrDrDeqmmBCtPMa7PSSjCyAV/EW0fsmArcHrHXKeKWTKXG9sDL0KBqos1kB vasu@vasuSrv\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQConeaJ2kvUOMy+jeKvXZTw655L3KK0ZG0HONIySFrRNkGa/M/hYf8oMNVh9SEax7vLNme0kwk7d71EhgtMGNsvHtQOkshWxM9P0XPk4HgB/OL+2LQ/7M3SwD4vuXPbqvrQS63YMoxcHBue5Ol5hXmM+qSoZajpUrfB2woqbp7aKsCpoK4Cgj4XfLHfh0n239ae38DU6GoAcL0GzMsxMdQhWzBwOO+ZHfowohc1pfY0+xGIMW1OlMN1eO7kuK2gGS/YRVOnLw9uu/2NErIjBzCihNN0PYWur2JE+KmU1x6VMaky49cKmAlc5WUqZ+FpV3gvnb3cLjBClnCkdjIcLeRR vasukulkarni@ovpn-116-227.phx2.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDKRFyOO7qitrq6Hk2NSl28h4laRTxA2zezy1wwjTqs6UL+oZjNRYQjeWD3dDNPZrUmc9JyI3j1EN11cCsKAp8YXz3ItB46M0BPeJ4TXCpg36vxs7/twSDMq/EoXA5sRBbUJFf+bthcNemwG1WsJoxRVzMezbWBTHvz5i4Hbh4sL3PBbac3FhT/3b5Q4tKdO6bXGXmNLAoALyVZZMfoSz9CWzvrO+g8Houb9iJiaXHp293y+ND9Wicl9wHPckO1VcocZ+ldByujgX1cz6m+pxmf6 2019-11-25T09:08:35.633 INFO:teuthology.task.ansible.out:AKY1OMJj5PjmpN0HrgpV2QLVA+ba9qBiDwi1g8sRHXkOETMZ2jq/2a/mmTtvrsF vasu@teuthology", "key_options": null, "keyfile": "/home/vasu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vasu", "validate_certs": true} 2019-11-25T09:08:35.931 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUqFGbkDz4L4BpIs2znPSWeawZZM2FBLrR3htaMBM6lcmiX2h+vgVGv2ZLpajGo7KB6J59kzWLBCJXN2DG0uUqa7dOdjEDuDrjZkNgDzXZ20lqzFAjLwNF6a9PcDN/oX6jzK9+3xArleQuTIJsOPRue2dQYGRnyIIrYpapJkkUgI+lXfwrEixCY3xjGNvVoh5bqQmkk3buLIvKEMwhPXJ94SGqsNTNC75HfKhVMjxA8QmuiXJVEgiW00V3Cctwv+yjiDMts4Dcs6yEzxXzjv0c0APxxN78bKlkwFgXiqawLiApk752/x2teSg7kvdrlGcCAWq/BhwphYjwaRlXIj1N shehbaz@donjaffer", "key_options": null, "keyfile": "/home/shehbazj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shehbazj", "validate_certs": true} 2019-11-25T09:08:35.965 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlOImwCV0AGu5L29ABaefV9Vqjv43WYJYFuLTxGQQdeQGcways6iNKGIX/nzf3UWkwgg25OVSpzB7zf6FjBx+HcQU+RJ7FLzeaz0s8ZxBXqWwtOYOvP9cXmAXA07hD5UAfxZKC1twBzL82bKJ2ZSmzfQmBjvpUOqRs7XwsQIClOW/ycY+jLgLRLFFDxzADsgnn6UEGlHeMETYocUbe89P3wbY5fHhqZmn6e5ncuAQMsiLRa9d+QhSdpD07Y/P65EzqRF4UOX+zpOc6gPlgHFir9pvf0bTbbBG/qKN9+KGqzZMRJGKenpHM6EMfLkf0QmrT4s4zd4ZnUQPq3HN3pul3 redhat@unused\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQD9n4ege98BNmvgUxogdcpptMILFfEisAQaiLUvSKpTsoy2gF7C6xSJXgDaziT5dW4ad2El4xmKQ/e6/e14Vx5i9pGbJz4NvL36Xmb3ent2PtY18i6d3CbnmGI8J7YH4YS9QvR7FC/KuvwLR6AdJ+uQShGhqG8kWJMjbqrtaxu8l1hq8ymBXw3O1Ht+xrZ/r3dEkAIVF3bMCSUnZpEbSsNoeiydEQl0ZCw+meXYmSw15vYVtKpYscZxqADqhSJJxIEHZA4E88dq6aYdV2AZ/s4cJK+tBl/ckBnclI3VGHSGn6oaHoO/fCHtKiENplDuKoLGQPO6xJ2Q/apHSGD7McY3 redhat@network\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDn/XgNfvqytLHgfBUdQIhJiWBlj+JrXqGZYvAZZJIWs32Ugm1/lCQTssGZrEj3+Qv81jLfkP6VQ8gXSWJkzUK4csxRxkZKXqcYLDE1csdmAFDOfWUFJwhMsCiiCyBiW9oapioLpvd40nw8k9qM4QuzaUhVonoGXMDDL1/dCgkbIjiQxxTJPHDEJFjtzul+VuhlhHqE6S1syoJG+UKo3Q1tOBCs8rwaPXEfCXFsyv+ZmcEv1+mpM8QRjdr1D1hfUfwb4CHtaa2AE6B5wL+zoY6+jsrTwZSwCfeIdgEZ8e3V2iXP5F7G6eXotu5r7mGYJClHZFLSfbNPN5J+EbRORl9/ wbhack3r@dhcp35-24.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDHYkp1nuRSYSAl6me1xuoTU30DW2YpAR6EH1c2adUC9tNtTZbMnab11aitQLAtZb9SKsRgnoB5KpH+zJ0VNKCuJU227r9vZwcVUruSGwA6X7LOrSkK0gFqNRZkVoKhC9oGHSU0zegYBDxx8VLxSV3CE56b+KEosEo6iJe7TkXy3wXHK6am0uhQPEdSHtdUdD683CGIffdF1RFmAFnzTPn96lC9rLe1mIGO+jVQx4zIo3i33RFQLy++4DtQw6Lrgp26Xr85HmQekjgyiya0VkR9Ypy7fgF2ucBGegnOgZDGCCY4G2Ruq 2019-11-25T09:08:35.966 INFO:teuthology.task.ansible.out:iZyKz2TZaOecDwrLkbYM7EGp8up5uLFRZsV shmohan@shmohan.remote.csb", "key_options": null, "keyfile": "/home/smohan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smohan", "validate_certs": true} 2019-11-25T09:08:36.285 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-11-25T09:08:36.297 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDZePxJmYerihzA0aQbMt7K7bctPEDcwPHiO8EVuJowp6zwhSmtDnYuFybKJsee4l8+rDKR6T5Ugdl5pLjSzQ+Ba00OSH7+qWC6shvVDYju5mIooJnc+kwx8RgapbGtDdl/yYskBb1KD5g/3+i/2aM+Igk+7nQR7Cny5+WnUEvHd7PHBbUXQSuu+mlUlnXCVMM8g8qlc1I3WQL5eQoIqGjSWRgZUIsn3P5oryYBzbuG9jqLwbaohJvZRJqkimQDe7zCFk6bB3z39pyOWUN+sldRvyL8FPyEmqog4IwpjcYyCuCnPhnb6fNq9yB65RUIde82Kkzx3LZrYOXTrdHgD/lr abhi@trusty", "key_options": null, "keyfile": "/home/abhi/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "abhi", "validate_certs": true} 2019-11-25T09:08:36.638 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJorrIT68EZc7A4IHwQGv8uoblSAIUnMxhOq6rlnvmqx4clpUzKkbzzrZUctP47aeqaSX5Mc6mV8TY0sIH8fFQwahF3ueTKT3YecZvPwiK4ALmawkvFpmgnJKKYhTrEApbFBhtH0KnNLtozpl4RJds4p09maNeoUXJDy3A24tpXViU2Nrw8lSIT6VJzvqSlK+miNEf3VKI6gnRB7fGcJe913zITf09Rew7GexAyQUYHusZbJCf5rl6hkewkZMkgbXfG22f/XWcYjoRtA/ZqeWIP2UDO/H3tfJnlMxUshbDGLllMFJzwmwhkYbapqVmnu7Y01rEQ4LqvaHZMr/yKn+N fche@very.elastic.org", "key_options": null, "keyfile": "/home/fche/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "fche", "validate_certs": true} 2019-11-25T09:08:36.650 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUqFGbkDz4L4BpIs2znPSWeawZZM2FBLrR3htaMBM6lcmiX2h+vgVGv2ZLpajGo7KB6J59kzWLBCJXN2DG0uUqa7dOdjEDuDrjZkNgDzXZ20lqzFAjLwNF6a9PcDN/oX6jzK9+3xArleQuTIJsOPRue2dQYGRnyIIrYpapJkkUgI+lXfwrEixCY3xjGNvVoh5bqQmkk3buLIvKEMwhPXJ94SGqsNTNC75HfKhVMjxA8QmuiXJVEgiW00V3Cctwv+yjiDMts4Dcs6yEzxXzjv0c0APxxN78bKlkwFgXiqawLiApk752/x2teSg7kvdrlGcCAWq/BhwphYjwaRlXIj1N shehbaz@donjaffer", "key_options": null, "keyfile": "/home/shehbazj/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shehbazj", "validate_certs": true} 2019-11-25T09:08:36.952 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmKchKXS6xKpo+25ZN5vMpbuL4OoFZWPfPSxur6iaBKdr6LSMejnAwkJOhfhH55pZLuAglHWdMcZi8H/ktzsZVd+ybvJNhpZsXdefNqqxUyjn320cT73pDdDC/W1519fUiokss23S6B4GR0B44VrVKezN5Dyhf4sATN8l76QnIALVK5hvYOI3v5k70weaPJM4fma7xYF5b6qSxauGofzMb5XwjNaaPE9F8U08nxGk5XK+S0rHoFnCjoKBlN9Ee1gk6PjsStQAyK5ujWvPGtG/UvuU9LewVbt6wuPbpwsf7Ck0u4wHECHK+185JF/X+qsWwv030pKShXblLVckNVhAx ani@orchid", "key_options": null, "keyfile": "/home/onyb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "onyb", "validate_certs": true} 2019-11-25T09:08:36.975 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDcsBe6CzgWh64iZAH8zbU6ra4yEiUBSsiUn0AHzyQurvjeRpL2Mut6B1WEi0kJG7BAM6b13JrhnD9BW9yIMIUIPpVTa2o4zaXP90+bvjGrgDeJbxv3T9/TGh13k+WWyampxm2lo90/+Paql/HObQyyIv7Wan0DOdIe2/Ue4ooWH0tu3kj+3unJTuw82oEDEn+9tGs0oleG2c8XN0gQyVl1DzR2Gqf9PCzXK7hInC3niY2huoSAZzVFEgsniKEWqTnVWYi49X2SsJhUyHCj+mqrTG3YnW30nha3EXOWIbQGblXVoTIOc+gNreqdGByOMICKSnQmPSG4DAl3c1Y4h4A71T3Q043IUv7v7S9tAEKq2h6oJ9I/a5z0u4kaL/+B7Cw7l4bitTdcZ0rcqSvOkKUNNgxujbucMSSc4Tsh5L0762xaziD9nQZl5bteb3VtsMAueqhu7L3sJy69+HGg7oF68t5TguErXfI2UNVGOn3eRj+zN/zSwLUXMoROlpM5yHXSzHvpsbgZiHBImM2CeG4qaPtqzFw7VH6NWJv4ypb0htOMWsbXmLshNTca1KXrrBcVozSJC8wKgEOxu4spk26G8x1IVTsi2ypyYX+irtAqZrrpIPdQ0Ss9NjKPhFNXAxJgLHj6TlH8UNh1gRfN/QTyzpFSe4EH6uOlnWuxcuM8ZQ== cbodley@redhat.com", "key_options": null, "keyfile": "/home/cbodley/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "cbodley", "validate_certs": true} 2019-11-25T09:08:37.282 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-11-25T09:08:37.343 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDJorrIT68EZc7A4IHwQGv8uoblSAIUnMxhOq6rlnvmqx4clpUzKkbzzrZUctP47aeqaSX5Mc6mV8TY0sIH8fFQwahF3ueTKT3YecZvPwiK4ALmawkvFpmgnJKKYhTrEApbFBhtH0KnNLtozpl4RJds4p09maNeoUXJDy3A24tpXViU2Nrw8lSIT6VJzvqSlK+miNEf3VKI6gnRB7fGcJe913zITf09Rew7GexAyQUYHusZbJCf5rl6hkewkZMkgbXfG22f/XWcYjoRtA/ZqeWIP2UDO/H3tfJnlMxUshbDGLllMFJzwmwhkYbapqVmnu7Y01rEQ4LqvaHZMr/yKn+N fche@very.elastic.org", "key_options": null, "keyfile": "/home/fche/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "fche", "validate_certs": true} 2019-11-25T09:08:37.644 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzf8fjotuelTIdz7clDpSN/VDtXirYrmzM/r157Np9uTMgn/rhzzplp2DKqFrCEDwawJ1O1Fr/H5DqttPonyRmjPmAIf96P8KKLsC600R63xk4DNhnByY0WBU3o1e46LP7DJLsOOy6hZRbyjgoFfzcapH7aV+fa37k7/Ri5mpmdmENa6lYI38+YaItFJf0kTXdFou0iLAt7xogOTimgblMst6UjOhRIsMFA3QS4PvOFR+AedCHeVtYK9mAeRtQ530wKzjz8x4NFU3Y93jmygwTTG2RTEfRjfbObfFOQXJGKcl28pxGPUL9ZlrLx1juAekn2Uz+2oO/eQ8G5w+XW3rp", "key_options": null, "keyfile": "/home/oprypin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oprypin", "validate_certs": true} 2019-11-25T09:08:37.669 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDmKchKXS6xKpo+25ZN5vMpbuL4OoFZWPfPSxur6iaBKdr6LSMejnAwkJOhfhH55pZLuAglHWdMcZi8H/ktzsZVd+ybvJNhpZsXdefNqqxUyjn320cT73pDdDC/W1519fUiokss23S6B4GR0B44VrVKezN5Dyhf4sATN8l76QnIALVK5hvYOI3v5k70weaPJM4fma7xYF5b6qSxauGofzMb5XwjNaaPE9F8U08nxGk5XK+S0rHoFnCjoKBlN9Ee1gk6PjsStQAyK5ujWvPGtG/UvuU9LewVbt6wuPbpwsf7Ck0u4wHECHK+185JF/X+qsWwv030pKShXblLVckNVhAx ani@orchid", "key_options": null, "keyfile": "/home/onyb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "onyb", "validate_certs": true} 2019-11-25T09:08:37.988 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-11-25T09:08:37.990 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-11-25T09:08:38.021 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDrtHyRuOSEXAM6mwL4qoM+NM4hpIKhaRS+/Y6RDoFKszkXFw/jC9ZYh7q39rQpWQBKHVe4RyPN9yyGWydFwaR/BjZi14ippmYOqVG4f7b+k39NFbMJRcSxYC+NvkqxK7+zpsnIWC+RdJ+blZi+zCbaaAYcMDSvBZW19FsJSLrO7hh4R1cuzgV+Xrmwdc1/VH+HqPMu3tHGv7N0SfsklWI8Inr0aiL83QqqgKPcVxljSWZGYvvFyTQOrkNQeaLURJNER9WIIac9o57WOINw8kR+ojebcOUBsrxZH9j+gMjsJ752Rph03wdMUpFcnS6yfvI4Ial7x/sGms4FE+wL/vVF mdw@pika", "key_options": null, "keyfile": "/home/mwatts/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mwatts", "validate_certs": true} 2019-11-25T09:08:38.317 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-11-25T09:08:38.318 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-11-25T09:08:38.320 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-11-25T09:08:38.340 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCzf8fjotuelTIdz7clDpSN/VDtXirYrmzM/r157Np9uTMgn/rhzzplp2DKqFrCEDwawJ1O1Fr/H5DqttPonyRmjPmAIf96P8KKLsC600R63xk4DNhnByY0WBU3o1e46LP7DJLsOOy6hZRbyjgoFfzcapH7aV+fa37k7/Ri5mpmdmENa6lYI38+YaItFJf0kTXdFou0iLAt7xogOTimgblMst6UjOhRIsMFA3QS4PvOFR+AedCHeVtYK9mAeRtQ530wKzjz8x4NFU3Y93jmygwTTG2RTEfRjfbObfFOQXJGKcl28pxGPUL9ZlrLx1juAekn2Uz+2oO/eQ8G5w+XW3rp", "key_options": null, "keyfile": "/home/oprypin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oprypin", "validate_certs": true} 2019-11-25T09:08:38.594 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-11-25T09:08:38.701 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options" 2019-11-25T09:08:38.702 INFO:teuthology.task.ansible.out:: null, "keyfile": "/home/prsrivas/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-11-25T09:08:38.929 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-11-25T09:08:39.034 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQD3vdP9QHyNFUNPOdplezEbdYsQfcC91ncATI6Fl/xHifmUbcvjIRqZHkCY6FNWUpMeg9S3+wlHJtzjVqJJ/MlQML3EtwnzQkeWv8bjtuZ07OqULYIv0zEx63XUfSnrdFleJ8+KDSpn7krkYE44q85B88IL+ZBeq5eCXFJ1TtmsDcvsN8WqZNWMdUGqgH3Bb4D8+YKrEfGxoievyxZ1A1DnBHan2l8foTKYggRrnuSbTb59cFpxY3R7Arc5Cn9kDJT1nDVBA5SSGCfRMiNyBOUzQVYe/UjKTaCpngglXC3wYnjoGvGcpksHLQCxyuJPB6djC1tdF+3Jehdf8y8+2TBzWXiiPq2XwS58sER5GcLLLQv26pPL8hgu6CdI0sHxgcgQBgpjoLwUMHmv13CnN/AAcZY4RRDK2X5zd6wFSEhuuCoUqh/m7K219IxI6fVkLaApC851D/ua/pj9OMjiOaJ+D13cPrgfqxva4eK8FQZvqQm7aUK/TT216QNYmn7hkzNjMTkjv/L5yYQ5Ym92AHy44wHzLPrtDB5mk94vHOjG+KlAt9ofuLncMSAXWe+fOqxuIQ6VfbL2LUv0EGYhHe4XOVj98q5utsVQxQbXAZzNQaq9FWXs2A4xcVP7aWj0Fooi17LN/W0ng4Iijir5hyLr7k7M1G6resgiY7p83biNow== pdonnell@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQCpgLVt7Zphmf6nqD+9BC0N91jdmaY6ZpvrYWlzbhxXITQ6juIEtegnlZ+8ohdeEfHFBiyu1xOSQSM1X1ugvpPQE94UqQg7JVb0tuoUCNJHYnIuv8T4YgyP0/vEV48k2G0BBWlP+IJ5f1iDvZuLHWLR5JSgGqh5ELaaHr7Cbyb6JlImldb7SDIoPHMCZ+n6dLO8l5Zy7qWelw6aeA770xmPucBr6t/BVns2g77XgJz6zFd99yEaDy27i8jfzL7WqR9i93yVE9EsL7tuOjg6G9KUzxBF0bg9iVsJXypvcBuG0JwQ+SDKqYw/cN4iLy5qBa6Siqrfpc/LKU21VNmJ0e5eXNgKKbMBpBb8GVmkoLaSRcXw7LclWopjNDA+meUhDAwU7b97N7cqN8BYhBKQyiShXFHMKLuNo2fSuTRt/icfb9VisKNGBdvwgV0oOlrqbXMzZKmj9amb9b+3JDBTGB7ioCsj5x/Evf9srXRrQ9kikIr1aBbVXIWir/lasNWVydcmUYsNi8L7W0vXArLwp+GgOXJFeZUiD3z9LG4CouRxMSwoj3lgWeL2ZR8feH4D/SP/+QmBdU6+aG97au9qreeXi5X8QTR3LND0DwwEKczX+3s+Fa950Oieom/gXC57PVKVh90KVoAGw3HzJTlZrauOtLkYbTUAX6woOMhAtJ5oWuDDT4UrHQj37jRrrl2dETLwKwUJN2+2L07vnnOZ4ABcGWLsU6d3bXeUBlw3pdsYqPebDKUEpjL2kUOw+z 2019-11-25T09:08:39.036 INFO:teuthology.task.ansible.out:mfjEfdVx9BfrvExvDX+3MkxccCJ8FY7od1s5fGEFHsXnIiMnLO/HZb/JN+Aoe+/ExadIcYwJnWcCJyly2/532kGFUlzOliCwPxgJpjWCKQAv6LjWmxoPXiA4JI0bp6y7lMEjmV2Bp3mwphlsW/r8Ie5hg1UPh/ttHRp/VZgcV8e6nNmP/qc6k9b2DWTNFqtWqm1hXLOwq08HzUEeH9mTqvenyAbv2RFMOS18Y2K+4BKUzHwZujrz9qEh/kqtpFScEIXkOJ+HZz3mI6XfjWFIPS487yRGfUvW3+nSWhvHc8vJY21bFRKdjD84qCVlZ2BfOCGUZSKftVItEiqXMTRs3RZd5epfCVZbJuA0hG16jUseeHqqZqDtlbLkzkxtS6v3iGRnPh/Ke/94FHHCzvYY8cmMYAVaM6p/G3edWftbaEtN70Fo7VKxPrpPzuSRZ7Vxmnb+Zad95DxxAC/vtMy/vkBVIP6dyn4HGxqa1oECFMe4oXdhnxZQoPzvWxb8upcXeAeOMMbcIlMq5a+dCBPGFNKhNhrb+9uAEDHwjE7rZlMJVSvt9v1S4hll6jiYIap+7n4reSKrS45aO9U0xiKXJJDohh pdonnell@icewind\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQC8MdUJzIvTCn0fM+hv1HAO/Lwjy/C/V7stQRWGIxyrRHbniyX8iuaU3j7YNFglSQPJq+YRM9UnGauM9/0J9tHKLsMp+zRQYrtxU/ThiTUN8TRJY11mGUol4yLsqkTtm6oq5sl6o+LfuA4hzaYc3XLrFHRJuCUgbtPG71Q0C/y3U+itrDK4wXUxBa4YzZzcIMg+c88VXwoouPbiL0gW5PCXzMaNtKlOqBUKlE935MLn70wIEfN/EwAAbss0ga1moqeqmi9Nlvm20DNIVFmkAGtYoq6CHXtL/uehzai5Sueop8Vmd8+1U8gDTo6JpiagdD8ZNf0zAbRx587KAT5ZMnahJpO276j7RNa85ZFkNkEnBLqBHRkCkTCYq1GehYyRaR69uuKyczFiIoTfAQgPh7VlnpT9525b/4nFsBDOkWxxWAIJR61YiKemPV/n/e9SwhvVsMDb2B/RBriwR7Mi2SZ/lBHGB+z0OA+EE2rGZy4LFVOfUczqIdpltlNotXan4M4F361i+bX19KE+f2RLySn3+GiFv0CdLwslS/y4xysz+FWxw6B0Y/EkpiDU3CdWM61+THGGm/DVhfL/Pvca7k8PonHFYw/x5w1nBcADpkM9JF+m5eOnp+weTEzl58sS+3GnoAuL8YFJwMW1RmE50xM0o5E3SDc8XftUvaSZz8mhII3uwDDukbELfmkLbnIeipQKyjO+GEx0JBp4Itz82qGuao0Wi32U9wHzhSWnX4GQ0pKZnq9frSlyCNwHu/iMiKF8sA2bX3dvplFJ9nKWPefcU8bdl/upr5hYrd2GttjUSkWZa+UuF264yx3ztcN7jotRhEmmdEcHVzKQ26/NyRI4/9DQpDKY7zw4O2wFkXWg5L9kodXwWzgy4cyOr4dMD9LBVLOpHc0CAji5/+lfuYQbu1mwVdfOzCXDqByjt7zCbeN7KuUh403sJ8PTQVp642xA9i4AkPv5g+fqi5C0++1DbZP0lmLfKtTswHhoN+kAqZu2N4Mu6mNVNKyZgW5h1pZzNSGWYcZttyYmMCHhPketReL0LPJ+VUsqQCVohuta3aaSe44InA28GZsXzMj5B0el/84bnE7cN4b+rfGD25NT73JEt3RgPa9W9jqT1hwd/nf53BDKpp7DlHjR77WSWT4h4F7bhbM1RsGSqAfZjCUJT6Rq+IuM2XcrLDY8zIeArkqaZLXMzfGj6KJ7cQes05P5LYfY4osXiq68Gs/7gzF31PQRRGOYAoqMN7752sn0ZMcpnUj3mJutQcncRUN4+naTMjU36WWD7it0YuQBnb2IJ4iKjYLInw8pr2zKgWtgr1TSCM4xKp/v1/ZMmbB 2019-11-25T09:08:39.037 INFO:teuthology.task.ansible.out:mS+dTzNC7Mpkt8vdCNjzZ5KZ5 pdonnell@aglarond", "key_options": null, "keyfile": "/home/pdonnell/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pdonnell", "validate_certs": true} 2019-11-25T09:08:39.232 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-11-25T09:08:39.358 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDKzhyWb3PmU88m8BbgpEuBzYM6+x28z6aZwHSesiyfXMfF4Y7IZjtKaTxH0ph/O9GQCPhRCVhRd8TlgKWJGhts1E+VkQfHfT18CgmrPWzj3gifc2FJhL97vB1Pd3jksGnOPAikgYPHP46SVMTtyvXCkOUuo7E29WrkdpjawcqM1rfpw63OnzHf7Ar7Sqh/othB7ajyVQ92t+XWSEqj7K+NRfTaq5L5RBWbwarM885o8xdI5mimOZ0QjzKOhFDORrI+HEVlkutyB8E20nZgo7TUy+kYeLVhEqb3A6Pi2QbA9GFLT69kA/H9nebhj7Cuim9yrx4OKeZuIncG0UxSQF3QmWflAZHBs8L/jniTz4DJ5id5jR29tvGK2LeddCEMB1L2cHwWHlG0aX5jaNEJe+nG6H5Tcg3+M1RPM8twBkRQmS3UzAyX/MTH1kktyFUHn8CdBXe0WTuoFSvXLB6BCHWzDYZOt0tIV/xMtBwhy+XGYA4oZSsDU5m7VlSVLBGL8DbRX69Z8AGey2y3FoFu/roY6NmDhTxUedJ2ZfoKSwdZD6jjatdO1621nonBnJxrDgaXCWzT7HtnWcc3GwxZoBrqkvgpsx88f4lNi22OpYVPV6A4hECe04u6OJnHSWVk3ft3p5PaLNF8aEXbiIgzZKAwX3LxpvEDFH/t1sdihhpmjw== jlayton@synchrony.poochiereds.net", "key_options": null, "keyfile": "/home/jlayton/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jlayton", "validate_certs": true} 2019-11-25T09:08:39.597 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDLCJHta/5O2C2MXRegwo+bJUiqbkESr+RwoH/VIdb4L0JDlVsqV+K+BOBjLZy3vwzWCaSackvshoZHAwGvvfGXjtLbhAYs6QWWHODf/XWBvOW4a96fbTToNpV1dXkHAGqlJMYln3GKVD0o9b69iQUTcLILJhWQjCoTjZsvTtP9OYn9YLfuoxhaaU3PazZVhZYml4Ca/3ns9vNQ/m+v4nh82ysyflGKYozEq0pl8VH0Ydy/FtOO98PhQx3crrivu3qghubntVRQnULUmlDSFhMff+ukLVdj2U0AdSBatlJuaSofWUMAQkv0Ia/om9doDGv2TqkXhJnZdGaWJwmuh6rKIZr7d7TYTD6Uag3wJ1NKiDETRNtmMCLuniEJzPO4asBpSSWGyVatyrNmAgBb6XCeZRkRcYvTXdA5OESKG3HJeguk0XCrjosIEYcvxyye9SgA8U73jWdDUdh0yIHLWcfrgC2eynE1uBnTPNBvtnf9pKqMEp5opdRAUhckhPRxJQ2JHcYMAFnq7gz1bL2boL2ElZOqHBx9qgv0ZKnk+YXqhyp2EP2JK0urd0EHMqkGUsRc7D0FSrz+j4lSkd71sHu3tMgwIrgDR55wYMBBRXTrR8pGTMnGldDBaxvUCgWnlsTQJdjs9xCM5gcJzzgZgkRBmrTOUkw2ABre6KMRAmQ4Zg3HxO0MBr5rfV5glWXuDJXsEowLPp/NQ+3X7PIU9LAtKkcZPWC3MayoFdEdNpKieCgOfRAiwJ34fWz4ozM0IvENmUmCKz8+yDAOeupJJ1/367Rl3+U8XTsDNSsqcmsRdr8ZnUIN0662FGr/nHkVaDq79rEywt/V2eMLWyunF0FUUnbwBB8/J0hgIQPX1ncziuzPTF/YCTR6lgVEtvNHPI1xTQsaVQkF9Whu9Eq6Gz1cFq6H/abb1DLJAoO1Cy0HjoT8MjwOICVcsrqxGXsNRBsv4fCxOaz6+WyVWz2O5gpNKOLLmenr9JbiTGjamjfl9dDempRROSDjsJJY6TPdRb0WBuk3i6mOAsAYeBHoargi+a+dQ1IDzGdVY2mUXtDTA6HluZVvzEKnAR6QzuEdRLOrbteETRQ3cV9STXgscCTokzDNzhYbkLUIl/SRBYBdPMiR9TgW+m/0IJkBTEEUuDtYVzyHezV94F17vzpK+xbRdk3/WmWPScPDkc9tDp0AGLdZ/FAkNzGM1VE8G5OJ7OduogIdIQW6yR8y+CP1O4e2WYZ3M0IjJFX8pc/zBinUBNOBvgAzO8aSXTgxXbs/Q1fcv7htmJZ/3nnB4U6N85tq5oklCWulk3tIFI5pMGiQUoQCW8PH2nwasaKKtOInBZMWm8B2K0c5bU2/bKRcXJNJ asheplyakov@as-ubuntu", "key_options": null, "keyfile": "/home/asheplyakov/.ssh/authorized_keys 2019-11-25T09:08:39.599 INFO:teuthology.task.ansible.out:", "manage_dir": true, "path": null, "state": "present", "user": "asheplyakov", "validate_certs": true} 2019-11-25T09:08:39.692 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCijPApnekNl/dsY+fleKYstEzNkKflBEcCoKtjItUCDJCVLVFOsDBrV94EGvDWpr13aj9sSwpUzRbeu/vOW8ZT853HH6Dqg2+eoYwG6vDRprZb93nw6l4vtU59/M0yKkyi9CPp2+YUo31cCFzYWpq8bv2j1xBruDSUCUdn9nJrHiLFqA4QEwSpw+uVCjSKzQL+LjlKmE8Plfd+8mNCQeY/OFsjxAztwN+4M50SZCnOVCIwEXoYRNd82JOXMOaqZJazSWTO8Tml6+Nu0hyEcvHkHxWRDSM+6LmXNnjlVUPyDN2h1/VAYATpOZ2VhmRarDoftCp9Wzi7eQdL/CGiMyuD rzarzyns@redhat.com", "key_options": null, "keyfile": "/home/rzarzynski/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rzarzynski", "validate_certs": true} 2019-11-25T09:08:39.914 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-11-25T09:08:40.009 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdGthG4qx90VQ8mJ7TI/IFFzptkSqBHYZsFIVu06IQgHbnFBvjao6heGt4UlkWJkegPCDWpKVAe7wFuXDpPlj/VssRHB5LBnYHKGfkNAzMNiMjpEZsTdy+9w39CCujipz0AlOSfrUdANBZ11AnG9VgQSPEukX5487juob55lFM6J3x+v4opBF3rSlux3+thbr8V0w4oCHMD9/aHvVjFoBnxo4oirR4jc/jgC2fQCMiAJhFKJGhjkKFiNHxcB3npBuRnpy+59BcScOAzSXdJ7DGnwxQLjymnhqsAcWinjjJhOjaq6NaoeKv5I7zQLaVGaQdIyuuAK1Dnb7trpfQU7mR rdias@rdias-suse-laptop", "key_options": null, "keyfile": "/home/rdias/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rdias", "validate_certs": true} 2019-11-25T09:08:40.246 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-11-25T09:08:40.372 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDLCJHta/5O2C2MXRegwo+bJUiqbkESr+RwoH/VIdb4L0JDlVsqV+K+BOBjLZy3vwzWCaSackvshoZHAwGvvfGXjtLbhAYs6QWWHODf/XWBvOW4a96fbTToNpV1dXkHAGqlJMYln3GKVD0o9b69iQUTcLILJhWQjCoTjZsvTtP9OYn9YLfuoxhaaU3PazZVhZYml4Ca/3ns9vNQ/m+v4nh82ysyflGKYozEq0pl8VH0Ydy/FtOO98PhQx3crrivu3qghubntVRQnULUmlDSFhMff+ukLVdj2U0AdSBatlJuaSofWUMAQkv0Ia/om9doDGv2TqkXhJnZdGaWJwmuh6rKIZr7d7TYTD6Uag3wJ1NKiDETRNtmMCLuniEJzPO4asBpSSWGyVatyrNmAgBb6XCeZRkRcYvTXdA5OESKG3HJeguk0XCrjosIEYcvxyye9SgA8U73jWdDUdh0yIHLWcfrgC2eynE1uBnTPNBvtnf9pKqMEp5opdRAUhckhPRxJQ2JHcYMAFnq7gz1bL2boL2ElZOqHBx9qgv0ZKnk+YXqhyp2EP2JK0urd0EHMqkGUsRc7D0FSrz+j4lSkd71sHu3tMgwIrgDR55wYMBBRXTrR8pGTMnGldDBaxvUCgWnlsTQJdjs9xCM5gcJzzgZgkRBmrTOUkw2ABre6KMRAmQ4Zg3HxO0MBr5rfV5glWXuDJXsEowLPp/NQ+3X7PIU9LAtKkcZPWC3MayoFdEdNpKieCgOfRAiwJ34fWz4ozM0IvENmUmCKz8+yDAOeupJJ1/367Rl3+U8XTsDNSsqcmsRdr8ZnUIN0662FGr/nHkVaDq79rEywt/V2eMLWyunF0FUUnbwBB8/J0hgIQPX1ncziuzPTF/YCTR6lgVEtvNHPI1xTQsaVQkF9Whu9Eq6Gz1cFq6H/abb1DLJAoO1Cy0HjoT8MjwOICVcsrqxGXsNRBsv4fCxOaz6+WyVWz2O5gpNKOLLmenr9JbiTGjamjfl9dDempRROSDjsJJY6TPdRb0WBuk3i6mOAsAYeBHoargi+a+dQ1IDzGdVY2mUXtDTA6HluZVvzEKnAR6QzuEdRLOrbteETRQ3cV9STXgscCTokzDNzhYbkLUIl/SRBYBdPMiR9TgW+m/0IJkBTEEUuDtYVzyHezV94F17vzpK+xbRdk3/WmWPScPDkc9tDp0AGLdZ/FAkNzGM1VE8G5OJ7OduogIdIQW6yR8y+CP1O4e2WYZ3M0IjJFX8pc/zBinUBNOBvgAzO8aSXTgxXbs/Q1fcv7htmJZ/3nnB4U6N85tq5oklCWulk3tIFI5pMGiQUoQCW8PH2nwasaKKtOInBZMWm8B2K0c5bU2/bKRcXJNJ asheplyakov@as-ubuntu", "key_options": null, "keyfile": "/home/asheplyakov/.ssh/authorized_keys 2019-11-25T09:08:40.374 INFO:teuthology.task.ansible.out:", "manage_dir": true, "path": null, "state": "present", "user": "asheplyakov", "validate_certs": true} 2019-11-25T09:08:40.571 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-11-25T09:08:40.707 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEEjb7yrcY6I6XbMU0hCps1NqBuYmA6HDmMcS/l3i5wJRBitIXYVlv6pefUG9gdBYR5NneccLfVFURu+DC63MJa2ELJvVXsQ6YUOX39bJUT9KlZs8mH+nlTD7tlPmP17Ae3k8sT0H42Q+/5+b5YvdAU+03aPhzntmy38SyvLKrvowRk63wHgcE9p23LY9tY3+HSJu86cb/IdePk1GLjnrMeT9aM+5j2eETrmBjMArP+/2UIsT5eb8MBkQzJ299SCC2gfV/qIVkorFl9xdejctAwNwdT0SPTMxCHxaiK3wZ0e/5HHTbSgR5YlxOlKYq93XBVYIaFKPCePk1At1a3Pjh cephalopod@octopus\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDcrTZoKO5e5xs2PB2UuAruvGlJESTHNAtqipbC8yJtRg8WGVDLt//rGu36QNSbpbJYMZgqgKsXBYolJB1ckqLYLv5j8RAoVwbW4QqKaV95y7x0bbXs/YAPnRBhEO1nfpT8GHQusUZvTbz38FDaQwstsYoncmsyZwBrqus9ycFavpY4wfpKoWtzeiK5Wjq7ervThOs98ZJKXJXOOxeI7mDakE3Jv7q2tBNu/rU1kzxIl65kWURzgxox6EmIZXQK3FzKt0jjdKtApVYSlx4ODLPI4iyxOv+MbRe3Nnj6FDhfz8/FeP/U5wWxdYx2WslzZCOI1BJ0VDLu3xEpLlu4vPzp vshankar@h3ckers-pride", "key_options": null, "keyfile": "/home/vshankar/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vshankar", "validate_certs": true} 2019-11-25T09:08:40.886 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-11-25T09:08:41.032 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+CtnvyVRSkIvD15O/q7/vQmD4oE8q7Mo6oT3kqTXaEaQ7iDayC2biUC0PC4OztPlNJxPUcKK5DfWw4F79OKeheqUbZRhsCC3Ge2JGoXHmsf0kBYdioMGiAxm+f8M/v1KESSSxvEnU7o+oi6VtFchh4Dl/WZ54rqZc/oQrlGNTHulFAuyoIAlKkCe3N0GdHN54PB+26QbLXvOuqpWmjVFbcjgRC62mkZl3LuCBhviXS4Rl5/ZE0QIHubCwV5XVrsqvscLS64bi06W8RN2vL56Vn3N9TGV4d2dlbE1PaBBESoTQ7Q0iUN32OAZLo7nF/1fiPY3Xc8r35juya5k+KR9z adam@TP50", "key_options": null, "keyfile": "/home/akupczyk/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "akupczyk", "validate_certs": true} 2019-11-25T09:08:41.232 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-11-25T09:08:41.368 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDeUJcLeyfqIk3ErPoFqh9kOnvUPsk/2hfaYNC2wZLMOSTIs3zXM8xjki8eN6vle0dmmeeAdwF+e2hJ3tCFKsEUQRvMc7ugIBA7L8/DboShYu7ItyQ3QRvy9pi6jnA/tlHA1LHalvyhMw0yZ4kvPbrfNMhZzOKzvL6jX166rqLA5AuiIfdYMBpE5rYq11GVjm0uQKw4UJpBMooNJeovEfQlNBTSV+IJSTiGBpBD7SgsuW3USMeEF3fEgQOcnvYdv5Fbp/crloyLWYc0HTidkLxpjP7vqY7E4BuBJug0zIB7voR3MYq6rAeuWjByPvq6coDlUjnw/QlrvarCbjjSaHdaRXLj/Dj+QqJFviESBr64ZynhHL21Bl6rexW7AwYyPa73Erbus3dN0aVQQ/ApcDTGHT7KRlDQ2YYg05YM4swpVVnrIdV58FzLWkll+ypCK/+kdtk5eRSe/KzZA3s7z4d4WLgolFWs8Fky8Rx1xDGewvMJgFUWVUbcGC13RocbNSxXG80cbIx8L/sclX4lDfGouu2sUXhP19vq1rJ7gZa6XTPohXjPbFS+En2cdlGdKXhppdvjT4QWKNzgQoF7qgWHMqJnMFJ6beT/2zP34KoM3t0ulee04z88VSn8Th8aqvLlGbdrt5LrE90escSaIAR1vNKZ1hCFwGieSKoRtYJTCw== nojha@redhat.com", "key_options": null, "keyfile": "/home/nojha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "nojha", "validate_certs": true} 2019-11-25T09:08:41.591 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCmp/WxBoQxK/3bI3N9X4INmChIFtG/75NULYt9CAAnoOhFHe8xHMOU4x/YS6JdoIBJXusrjbDhCksFSK/caoOow9CQp3VWytslRSjfjkjbwwWHSdcyGpSO4lS5F32SgLOZBuyktpkLDgi6Ou5OTNKokJO8qiZo9Uw0rEBNakfsVi+fpysUDL792Jk1ROUr0HUpwqrQfTPZaue+PomXPQSYIqf06EH1nJYho6D4jt/g6U0qc66WTW5dXTkWD2VVQjZbFGkamz0ICHBJ/Xpf4TjWQVotsYD5NtbvqPo6ygpJYNhKR2n2zA+L4Zu2n0AEib5S4ftNxc7gNDCCfVs/EEIvXp/3CjKVPxHf4xUUPUZj+SQFnSUPQfeY1iIydoX+IG9oj5kKmQBdQFxbAaZJ679bORIqlH73dUFGr2rfS1UlIgPW5ll77bixsB0GP3+Jw4FF5QbO+k1d5qJpUw+8D6FbWAppP7hnSAok9PTpBqfkMpFiX00ZTIFwKCBLxdtJOfbIhbOPCnyEoB2ZLfRiBJmXhBpLjGghWIPPnGZh+yeJiy/EyKwN57vLmjdmvcDgSsjHswT+rQ+QzPoF+/KcdU1LL6ME6i+R7o7aiLOdDpZo4p5d1kr3VMNjVNM+L9ECpTntPhyrglOX018Q8xcNRh2/z+G6NsiiV4t0YxFdk/ICTQ== Jesse.Williamson@suse.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXsBAd8JfyMT352SjfhZ2Nj3gR2PkeyPS9yB6fKck5CW0Ns1EcUAOIFYw7a1R6zCDJbkkXZDRgM1W6v/UBgpOm69HaNCZuqx19mQRNgo1lhw6hDHc+9754wOmIbgu3gInIS/XPvGzAkGwBW3rWlOaNurY4Y/PH1QyiaujD1Ii9X2Q6yRP8hsDrccual36gngY6wV3G4xKbS8kTY4qQekHkgFT+ZDHv9ibliDfAxtzm5X8N25sICpvcnQkG+Co3QP2XADEhoQrs8L7QekKYPlX8zq6rJBO0cGYW50E58LJk7s67n7QQPCy04xeAf9KOZZBSMHTI8eSFHjb8B034IkHH jwilliamson@linux-z0c6", "key_options": null, "keyfile": "/home/jwilliamson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jwilliamson", "validate_certs": true} 2019-11-25T09:08:41.720 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC+c7zV5QmZb0WsgA2UZogoDNaoHcngRVLA8+BYk/U10ZlvCckJSfIXWMlBnqguZgdm/jBwsw5mGVBgHxUSqFNybwIlU00kyPZU6VbyVP+ZNVUf0XIE1NlemldgPBxxcOzSHkHhWrUqA3IKiCllcQVDQzutfUnq+jDT6NkVn4/cyNzNlargt7FR2L8gvskh62jLTBbhEuHMBp3jLCvaHJOk708gXm9qZ5iLOGV0pu7xdg58KE0p8ILClQBTbvoqPgZE9LA3qaOk1NR1n22kuW8JRHaKfD2Hn0Yu/MfKct783Bxb3mxWBKXxmM4otdmsR/R14NH/hIPce9wSkOOnQAC9 ifed@37526", "key_options": null, "keyfile": "/home/ifed01/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ifed01", "validate_certs": true} 2019-11-25T09:08:41.898 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-11-25T09:08:41.900 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-11-25T09:08:42.056 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDoEudQMGBPqfZBV8bXpdQOgVTDP6GFsr5oL4gdbHFMuJCGe0Sfq8LTaSEBQx0MH7tXc7JJ1fU51U7DSt0kfy/PFeV1Y5XnNATiZu/SzdO3jHe06/8CCr3DLnlEmzXASq+tqDJbY1eV011d/TX1jiC8Qd1VNbpdZYExEDsDzwJXnWdh6rjTL4anGjIgQWcmU0XdbUJXvRP4dGjVvZVHZnflAkxLPBbAx/8BBOuBmv6vP4hNs4poi4VLwR6S6Ha1fgMJmsn6StsCVuOvKBJFuGvaulHgHgdoDJ1ryUFikkXXCWoNx4IbwtpQIkSxWMuAovlwwk9G8ozSs+RrFzCavE0f omw@omw-dev", "key_options": null, "keyfile": "/home/myoungwon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "myoungwon", "validate_certs": true} 2019-11-25T09:08:42.225 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-11-25T09:08:42.405 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCmp/WxBoQxK/3bI3N9X4INmChIFtG/75NULYt9CAAnoOhFHe8xHMOU4x/YS6JdoIBJXusrjbDhCksFSK/caoOow9CQp3VWytslRSjfjkjbwwWHSdcyGpSO4lS5F32SgLOZBuyktpkLDgi6Ou5OTNKokJO8qiZo9Uw0rEBNakfsVi+fpysUDL792Jk1ROUr0HUpwqrQfTPZaue+PomXPQSYIqf06EH1nJYho6D4jt/g6U0qc66WTW5dXTkWD2VVQjZbFGkamz0ICHBJ/Xpf4TjWQVotsYD5NtbvqPo6ygpJYNhKR2n2zA+L4Zu2n0AEib5S4ftNxc7gNDCCfVs/EEIvXp/3CjKVPxHf4xUUPUZj+SQFnSUPQfeY1iIydoX+IG9oj5kKmQBdQFxbAaZJ679bORIqlH73dUFGr2rfS1UlIgPW5ll77bixsB0GP3+Jw4FF5QbO+k1d5qJpUw+8D6FbWAppP7hnSAok9PTpBqfkMpFiX00ZTIFwKCBLxdtJOfbIhbOPCnyEoB2ZLfRiBJmXhBpLjGghWIPPnGZh+yeJiy/EyKwN57vLmjdmvcDgSsjHswT+rQ+QzPoF+/KcdU1LL6ME6i+R7o7aiLOdDpZo4p5d1kr3VMNjVNM+L9ECpTntPhyrglOX018Q8xcNRh2/z+G6NsiiV4t0YxFdk/ICTQ== Jesse.Williamson@suse.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCXsBAd8JfyMT352SjfhZ2Nj3gR2PkeyPS9yB6fKck5CW0Ns1EcUAOIFYw7a1R6zCDJbkkXZDRgM1W6v/UBgpOm69HaNCZuqx19mQRNgo1lhw6hDHc+9754wOmIbgu3gInIS/XPvGzAkGwBW3rWlOaNurY4Y/PH1QyiaujD1Ii9X2Q6yRP8hsDrccual36gngY6wV3G4xKbS8kTY4qQekHkgFT+ZDHv9ibliDfAxtzm5X8N25sICpvcnQkG+Co3QP2XADEhoQrs8L7QekKYPlX8zq6rJBO0cGYW50E58LJk7s67n7QQPCy04xeAf9KOZZBSMHTI8eSFHjb8B034IkHH jwilliamson@linux-z0c6", "key_options": null, "keyfile": "/home/jwilliamson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jwilliamson", "validate_certs": true} 2019-11-25T09:08:42.518 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-11-25T09:08:42.775 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAAEAQDZPRW12u7LvaiynB87ad5h/JFtYIVjkKIOym9Y5CAbvatHhZ9fB1WAnYIn2HqiSSPUXT8sFy8w0TmM+aH0/ZGcfXoe0CdTWLtUbXKNlPI5h7TO9rHXESP2iRBpnKs3vK8WnMY0/D5BeESrPfwVScucmzaDrjUqjdZKR/aEeqVvDpQsdaCaoqNg23t4nhQ54BEvK8O+14nQ26Rtmm40QHO1ze+0UK+7nUMMFwxvd1EKjVp21EgYAFEbyQ5Kbbl+fmS/11AGy/yTai0c4qd7ohBzZvx8+sY+c1novG9Q7LhtyxuuYwzYIrGBCkKH5+iC7nt2rGBP+mnTfIgdbJUiXX3cdrMESwOSZy6x6zgbXXME+yfM0myhomKmUkpeZAQInX2KYrnEuPD9kteyK9tZcG6gJSaiUdvlab49Alx5lM2hc62M4s4WKaIgroCoApd8FtNAlTQsdYuycntgNrbPy0LXQ5hw8K8hkvc6RbxISdj/RXWguQSVbJWdS9Pf5NwD3Tq7dsOghJWhx4vBkhW3IECVrPQWVyAmgQ9oj1PoL7/2LZ6h9P8F0KC1qmhm+pWdEPLugOlU6lzQX7RJcukYlsINThM1RGwbPE4h9XcMyHAnODiDF63TDfQ01oKbVXj9XpOGY5eP0EQxKJYjCfWW0Ot6nK2ZCt8OYvoRkWMGnLv11Nub3NNOgh5TB/4nDvEU1cIb/Ikb4BRzVIEbast/pqUJmXe+GQ+ho7mp8y8a+lIxu54BR+VN6isjwODCVt1DYohgixEoul7p5pXvTM+JUY2GD/ddTGwPvVAN66sWnT9bjZZunQwTHqasFCIHNyPhnEnYIIqCo9O1I/D8zRQG3SYNxT6ED9GPARrE/H0TvdOmKaT3e0F2GRJRSToNxol9dpJovUziojOhXyBUjlTqnMY2RVilX90MnzgqR3zGO0aSruTGyNT2Faymqxn6T/50mSdJ8EyWH23BaaXUUW/z3Cjov+yCQjLY48SOWwlxKXw3hGz8xTAxauBYI01ojsVLCpCUGFyKlWS5iFwiLFtr1KPYqeCoS6CoWjiOiFgGe+w3lPPfp2h4rc6auIx2DZ1ByYhWLeHnpkV7sG+q5ckBIRAYmiKyXSGdV4zbXITMWDm+3zu4oj5VPRCfjMlout8OZ4TQm5PUdKZZhSE1jULG9tPDFL7nWwM3z0HET+d9kQYcEO6QB+ma9gZHp1qZJbXvCIC5fQ9y9El++A6N3dBr+ED10Ye88PLoWPeumyAmc24BDdI0skGk89Q0IzoLqagK6HGhPAso8si9wr84CEi4ncpizVLDU2cFKCpef4Sa03HoMbwfTgERtyptKFRbMT+SpuZsnrhF635h9uqFnszowUXV guits@Elisheba", "key_options": null, "keyfile": "/home/gabrioux/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user 2019-11-25T09:08:42.776 INFO:teuthology.task.ansible.out:": "gabrioux", "validate_certs": true} 2019-11-25T09:08:42.865 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-11-25T09:08:42.866 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-11-25T09:08:43.124 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCpDFqu0Zg94mCjrA0FTB0vMiSBml1pqYCDKHhKBrfaza8u3bhtsmls9i+6U/vIwCEVk4GGPNhWduTJK0h9R5MS2eCf3GJcE0XHqRp/8msUCDPOVSZwC7hoYHZKunUIUaIUajOnR+L2slA/a9JaqataFH7zdrKbPdSKBzf5CXYrc0fK+Fz5OOsaI9KITbM0nESVpNCDzRqnA/aarF9MlN79vpTtvwQ+i0z2Km3cxqwV70GPMuFVo4CGf8pCGpbMrIg9OtoSC1svQ2vZzLkH2WrbDqTfYlOEYJJXaiBKcGhGnLZXfLIGSgvHjpv6xHo63yaMihocyg2MwTS8TZ/VtAaH seb@redhat.com", "key_options": null, "keyfile": "/home/leseb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "leseb", "validate_certs": true} 2019-11-25T09:08:43.207 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-11-25T09:08:43.476 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEAzjJ6nlPn/OXP3knQ6orVId2QLswql9gRetKYlicKDBrdFg0MES03EIuAonu0aUR4NEReR7P1iB8eNnezrHyAu5mxIJlZp0rz7ljcBcTLL+11IJU9I9OjBFwzXGqJ1KK4B4Qfw35oc2O6F5V0JP2nHhHgyny37udJLImhxHy3RNmDwc86JhEBCVR3ouYiZuxgxxbA9BXc6+Lkdu9hEMMCDlaEXwNmMyASVU9Wf3dKnMTSCdar2PANquWbpNQZ5cgoTm39xgust5CEyYJ53aOdrti0DYK7qyzxUXre6ckZGrH4oGv2XrDaxpACIiHXt+slvow51gsZnBRHeayMAxQzGQ== hchen@hchen.csb", "key_options": null, "keyfile": "/home/hchen/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "hchen", "validate_certs": true} 2019-11-25T09:08:43.523 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-11-25T09:08:43.806 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDC3aWYoBDxRh13e+qver2xcmCtNsiVEt9NkN1cntdUxVbjfWgb+E/b54kPW456tswoVaYS+3xI5DYTPTN1tMhx8jnrgVKI2HhIYC3SO0+V+HNofzU1p7v5CfZxwyupXOgdfHmEWKOepPop8qWon8wZVUkj2t80w8Ifxh5gLVigGquc4kvMLyy6nBI0k6UGeRr9lSKmWLvzudWP70cWij3ynwxEeIuurU0HI9tDmjaVW5dJuu6IZHDhJrCpeUZBQIDqjH0pg1i1uXXwwiLI9NW8jWuALREDeMxAG4IgOZtoXdS7J9vVXHxKpa8ofb+qYNXI7B3OTLgjSS2ziiU1KEIr wanjun.lp@alibaba-inc.com", "key_options": null, "keyfile": "/home/liupan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liupan", "validate_certs": true} 2019-11-25T09:08:43.839 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDCqZKc+/7iPLhapA/7SYmLxwyKEM95D6eFfsDBNKbEUey/B1OlqYSo+yKB/LdFDyUcd7shMDnhFFg47lNMANrsZdtE4iMPVAa1YhcrTgNbV3HSp9cthKixvU/gkt9Fnb6PEckRzr19wEwxtoNg3Fh/R6YCiSjPwqHmArtC1G2hzk560EJpozx6xylMR5IafatNDoZVw2YQ35fZvHbhHD6meQELducHJ1bKEEHxVIRSTcmXDpBz/vAvBoQOEPuH6PmC/oSwL9HVVIsazf65F4QfK98d+8NCK9TemgXyGDk3Sjz6xdm76yBhFzDQEAzO12U1YtS7Q7zGVsjmSDav2dd7 jcollin@earth\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC89dHSsgjgtIjJ3e/pcMthliQlX2vtdXL7pk62nD64230Fm1PfOVCnzn4LqDhBbGgJT+BrlwOPfzStpLgnUT3l67S834SJzhvSTTpQp2pznnQg1rZOiXjvNhbQwVYV4rphXWqay8weWvFIb5QfXE66mYA1etZtcOb1SMY2ofAjUiA8bAQS5aHpzcMVs4N1A9x5GJx/7/Bj1h8zPpSXikDYasJc3++UMGc45C/AAf725YwijK7xMnZNXv8uTBoGzO/Ko24R3VdvtfMVn4Hi4/FwuRd5bRYcJXR/nUdtA3IinqvlnTpFV2LbfgVTHiUMGhvewRTB+8S8ij4nnAlko3H3bhqZZTk/y7d4K/IerY0mozVIc2x8rMLfqhF9OKv1QFB9fskRZcF/ENHITnmLqCJPg9rgmyv1gCQ5Tbds1OI+IYQIowdiwtaViiTSIy7/CedbHqMz6aDnPsJFMs24lcUXe2d8uZ26RqMinMFGXg0zF7tifuVmxj7zG8DT7MFORDm5f/Hhkng05R5XZesStTBxpEU0MGuzWhENt3RuJdstqmY+wkDVW3XwjUvKG/gVYuJYfnBAsRZIZezw2wfvodhc8EmeHELvY1YtB3DGTBXKE1lsKGN7yV5+nl4UtR2IGzBn9GovWVi5ooB8UlSAeUmZix/eAA5teL8ldFrDht7+AQ== jcollin@teuthology\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeApVHu5er+R9h52RLSVYVhXFQSn/nMN38uWoDYPw2bqYBLNEAAMcPl+rnqHubELpXTGXjJYL4n2mjic+2rOLZpDgPLmOOm4PwgYstJjE1KLegKqVzXIa9HfASKMXaHSgwtC6P5vDQkm2MNl8vXuTAITF7VYdDS 2019-11-25T09:08:43.841 INFO:teuthology.task.ansible.out:33Kkke6ViJRZI7szYjbP7z9A1fy26THrvnuBT1Q18yy9vgk/sKo8vjLQfHSLvO90CJzouNEKqmkVpmJEqCPbE+s6Qr4+d2KY7L6Q6GAXxrqXK1jSDrrPsJguE6zZ1oWy/7XFMOr9ssduFWocnjQjPtQj3M+V+7FYAsf5uqQnzeMQTjO6AXYxTI7 jcollin@stratocaster\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDk/DYRp5E8Dla/4XHCF504X20W0u7CijZSCFWT1/5082NgnYeR90Prth3p+4xsw3YC9H5pEAS3XwcbdUb/NtWvZVIAIcMd7NOOEClSIj2Agk7NJxZ/4SfQRLrK3H2Qli/MVnrTxnEa9oeS4zqzv9dKRwFJDUFCoH2CpoumFVAp02uyzk7D/+XqFrXzMJUiSHXfl3YM2rt/vNmCtHtVp04UcG0Kyl2TXY37frRlSOkMMu0zbIC551gOyae6NGYhkYHa7u2SnngXcKM78b8MDvMv3QtuUN5P4WpH5qFRdlKNKsQGrzcQpBDY8AKQ+rX0K1ENhBwer49bihYNMKvjYmvTdLTNWgPCu7dt8JSgJQ8JMMhve0pjvFdpcKDLp0fvXDvJvZdLJM1ID/66kywJspHjShuqQNXRMfd1R9C4F20c+R9LY6eTGwrpLG1DNEez1VBHmOu5hQl03T2PfaThfLfMHRTw8JpGXGLMQarCCTm1PC6Z2Iw7KzorxP1eofGA5q8XoCbwGD01uQRZrwxPq0SHzTAkW2C2M8oEz+k4Mrx7+vGYzSS14hqxrzk91wuYhZCAKuGpqwm1ouJ4wxvWFewjbxLgucMfGSp09zJ2XSYRmtm7SAUifFob6Z9czHGTQerHCOTPuYqzlap2gCCVYtEvQyNB+4IqK/9FtEXTr28x0Q== jcollin@stratocaster", "key_options": null, "keyfile": "/home/jcollin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "jcollin", "validate_certs": true} 2019-11-25T09:08:44.129 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-11-25T09:08:44.180 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDdxi9xdXNBYctMHH1cuuyNcPC4I4xVAeD3HZsKH+QwIILiKOTHu/6HZg7Yo1WsuwYSA0MDc+5LCSGu7RsmOEVpcSQ4gtNoFVwsThDyI6yhsm9WQqqXwSPs7ZE+urqd/BDfQ/7oMqJm5bViQMP0vmBExHuIFagj+8f6nspkE2Z7lBbaw8bEbAJZFx74oReOAELsL+Pbtf9Iaqrap4Z3WysmhfP6Rm/q7mncBBmo/fl/bejhThr8fghTPJb81M+XLhMk/rAdVrGdKV9iSt6lLiWdvf0lpTZO4LdIfJSBXr1Xv+lHRtaGXID2ahH7zl/W6dM4baz+hU7ZrbP+vdKg/SVH xxg@xxg", "key_options": null, "keyfile": "/home/xxg/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "xxg", "validate_certs": true} 2019-11-25T09:08:44.448 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0PusmXbjqduPTosGmqFsi2cUjRIiWmMnATY/vuawopBxRNykGnyeh1MfVLgkulM6zBJ3cDZxLsZYykpsHGvaXgLL5r6bEOOKPu8sSMwaTfZWNewue6xa1vOYAggeWOt8KTd0IO/pZZfvFcJ80tjtcNvY8zOJ9TO5W7RpglfJhPneC9Kr726yZ4RX3Hqlx+nIMzrSlvZVtwsKikanImvGLDulRiHqgE5BcqGvDF1yCSIq4OYF2i2Sk+dnPnQtN2uePsxPYKikd0B/rua5q+hDDAz2I36NhaUIhLteE6acWJK5Z2s0o1rWAS5XMbF4qYM3nbaE1Au/nXHcjO3sf6mj3 amarangone@macair.local", "key_options": null, "keyfile": "/home/amarangone/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amarangone", "validate_certs": true} 2019-11-25T09:08:44.532 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCsA63hT3ZAwQ9hsvfY+o2Gy15M1GRnRw1ZBSXEiCs9OJku7T9+LCvvH85Y/7V+mOGcsCZKqVTd5Vy3sHAvAF3M7CYJ+5IIDJKCYslC0wkKcX4jj26aMToAZPms8uvs527BNgKOcvwgIfPjs4WJG06fey5AhsuuL30etbAvbBudxRAKdZSTOsMveneKtXQu3Dx7y0rjfI/RBAhCAvisnMqYFUdajoJIl5gVXl6+8mauUOP5O0mvyvJH2jQveqZXTdsIPyp5mq1nA5psqvivWGz2bEJ+owI0GjfuYbyAi6i+BL4Bsd3GeoAs1G3Dicfr228VIUsT+R3xLBQUj3xIsVAZ pcuzner@redhat.com", "key_options": null, "keyfile": "/home/pcuzner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pcuzner", "validate_certs": true} 2019-11-25T09:08:44.791 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-11-25T09:08:44.893 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDC3aWYoBDxRh13e+qver2xcmCtNsiVEt9NkN1cntdUxVbjfWgb+E/b54kPW456tswoVaYS+3xI5DYTPTN1tMhx8jnrgVKI2HhIYC3SO0+V+HNofzU1p7v5CfZxwyupXOgdfHmEWKOepPop8qWon8wZVUkj2t80w8Ifxh5gLVigGquc4kvMLyy6nBI0k6UGeRr9lSKmWLvzudWP70cWij3ynwxEeIuurU0HI9tDmjaVW5dJuu6IZHDhJrCpeUZBQIDqjH0pg1i1uXXwwiLI9NW8jWuALREDeMxAG4IgOZtoXdS7J9vVXHxKpa8ofb+qYNXI7B3OTLgjSS2ziiU1KEIr wanjun.lp@alibaba-inc.com", "key_options": null, "keyfile": "/home/liupan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "liupan", "validate_certs": true} 2019-11-25T09:08:45.133 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIGGibBaZykvZPLLQm/UIDUlqVxeyJvZEOO+/7pjqX3O47nilNILLnv/oWhQN1S5eGVev9lnTlTxqI0Mt0N/B9vbCxoxx3ujsSgc+FZL58rMBN4uaTvHhYgqOYo3ht6fciRqSvcqeOGNKU8HnXS0/C8bJrYoKUTnmaaZjQ0bSw/AMsAIxu3bZmWT+HfgVaMh9PjPgC+XnDypEcRkOLbeEIs+0s1TgSR7Hs/vEuVtLH3Ph4WZwKVKQhfugmbblLLNvHvTy0q5afdh2RQ9MEaUgxXMgVq4hJ/JcFNZp+ZU5irOPQosZfjSLWJKx/LH+Ye4/8nINi08w0MZgcK79CwukJ kmroz@suse.com", "key_options": null, "keyfile": "/home/kmroz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kmroz", "validate_certs": true} 2019-11-25T09:08:45.250 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDaRX+aBLjb/h12ctmvoMvZro6NrrYDuxROF6QMVpEVucA7jHhwCC/j9dJAq0hOVBO8R3S0bvKQ6bmnCb/4y2J6tq18xKUFCrChNii5OQ+AgM5v1353oZ9SrHN8p9ajvS8w313o2OpXhpaPv3khVDjKZsDmeH8YL/Nx3WQKqIJC1mI4jrI0sZGGJLifEy3pPkO0W1KmguN9AwAWK6+kckAjMa3ElkfK/xuAqBYgNKQUxP9oMgTQRcgpWF8Nzb0npN3v+HOUIRrth1INSchkDKtWob2T+0Zqxwm+Vpi0iFWwqjcN/OVdaDGf34//A3M/tsiQ1g9SqhdhZJCfu5XLmM63 mkogan@mkP50.redhat.com", "key_options": null, "keyfile": "/home/mkogan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mkogan", "validate_certs": true} 2019-11-25T09:08:45.463 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-11-25T09:08:45.571 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0PusmXbjqduPTosGmqFsi2cUjRIiWmMnATY/vuawopBxRNykGnyeh1MfVLgkulM6zBJ3cDZxLsZYykpsHGvaXgLL5r6bEOOKPu8sSMwaTfZWNewue6xa1vOYAggeWOt8KTd0IO/pZZfvFcJ80tjtcNvY8zOJ9TO5W7RpglfJhPneC9Kr726yZ4RX3Hqlx+nIMzrSlvZVtwsKikanImvGLDulRiHqgE5BcqGvDF1yCSIq4OYF2i2Sk+dnPnQtN2uePsxPYKikd0B/rua5q+hDDAz2I36NhaUIhLteE6acWJK5Z2s0o1rWAS5XMbF4qYM3nbaE1Au/nXHcjO3sf6mj3 amarangone@macair.local", "key_options": null, "keyfile": "/home/amarangone/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amarangone", "validate_certs": true} 2019-11-25T09:08:45.814 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-11-25T09:08:45.929 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC56FiAgzz9aF+qAMkw5aGoCyoSA8Lf2LPaz3QxTYEzmQt0ByXMmVw0cgUI4gDt7hMBX6LoxPALZ7C1aH8aPTpb6+hxHGfir2tqQmU49JOQiUvBP/Xtt1ilxMou9Wu/ZadEhIZxaP/TBncXNls9cbFs9bV7Q8J3CLCTGRa8kilEpZ7209c/xQI4yQ6q+Q3H4FkO0YHP+Tkn9mg1df+BKkXfSBycjAyOXPvljoetjw0FHEbJ7lTCcSG+hCb2LEACo2OK9eWC3GYwDPKZF1Ht9HfDvg4bHfY3OVOixYiZSim7dbj3+VHWWsiH64usGH1o6fn4vV16UhVJiiicPMWvllzH vikhyat.umrao@gmail.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9G/I5iaw5d4ssyJ4iTUxSvIBAX+P03VVHsNMHAB70rVGGwBnBJFqJNbukzxfnsqwzN6vSeuMy7I1zc2VVi4GtBQPK0iCbVQN+Njq0VCIxb4hLmjD3Vn8NgRp80955VJyYGT9+/JbZY1UCdPBuld6hsnTipkdo5U3huAHD4i1FpYKol+4DCZHAO5wNilCbests0+ZPtHYe53vnNDz685cx/f7RTtEarvU2fECxkurlBBNvuIl6SN8bo+NaJ01Li0xAD+2fpGz6Iu0WUEdWTkt78ubvA4If2k8XAKxkOKEvfyX+AtTzdHxWSQu7ixkbuksuK6A0bnankn+DdiP6Y293 vumrao@redhat", "key_options": null, "keyfile": "/home/vumrao/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "vumrao", "validate_certs": true} 2019-11-25T09:08:46.167 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-11-25T09:08:46.261 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIGGibBaZykvZPLLQm/UIDUlqVxeyJvZEOO+/7pjqX3O47nilNILLnv/oWhQN1S5eGVev9lnTlTxqI0Mt0N/B9vbCxoxx3ujsSgc+FZL58rMBN4uaTvHhYgqOYo3ht6fciRqSvcqeOGNKU8HnXS0/C8bJrYoKUTnmaaZjQ0bSw/AMsAIxu3bZmWT+HfgVaMh9PjPgC+XnDypEcRkOLbeEIs+0s1TgSR7Hs/vEuVtLH3Ph4WZwKVKQhfugmbblLLNvHvTy0q5afdh2RQ9MEaUgxXMgVq4hJ/JcFNZp+ZU5irOPQosZfjSLWJKx/LH+Ye4/8nINi08w0MZgcK79CwukJ kmroz@suse.com", "key_options": null, "keyfile": "/home/kmroz/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kmroz", "validate_certs": true} 2019-11-25T09:08:46.516 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-11-25T09:08:46.589 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDZDKRAePFqjugc2ljw4QmM6r7B2dPI39OBB+2HKONtDZI4q+uwgZ1yqnhrbJaU9ux0W5wNzJyp2VfOPhw/9G9jo+efJPg5IySnTonhi1zeCXNpBMY70dJZVCyGEDdWKehvFM9xFBOxBC/h7gOyKqEc28BKoy4cHfULzm4er8PU30MyFBngu8NaoKqyQ3GYAfWOAXF5lUdalbJwl3a+HBb8PjUhbGuznAZF1Ji4JgnNuEZdlNVOKwhEkAFZ1JwXvgLfrEJVQs5b2V9Sp2Vm5ZngsLg7b3RzV5I4B3Lj7QTTraV1zLJGiS5iRdjY7D4W5Uxi2k1X7vaOjySJWWadHyeicNNBhLSfHl6JJ3aMuA8oAAmtgo/8IkJ96GFtgM9j31Wg8wkeTyQrwSeqCuv1uDFr8rp+OvMClhHnDgUklJLWjjYPCDVYR5KV+iOiHL58WJg/b4h6NsoP91wO8P8EcSaiMdq7P77w5nJxDI0KKb//dC8aHtWnRRRlrIS+PNPqKJSNTaSwOTieXvzplhzKFzu2dr79qs+/67iaNE6HBvIX5jVHWCFV3wllyjsC8WYPh4kEVtg2SY00SDOOrMrqANV8DDtHFwwxQTEjz1z5ESuwy9UcUSQfP1/zsAp2OQzPutKs6UipHyBK3pLm6MjaeeaEWu8fmetCqALxYV3h17O+Yw== miguel@hermes.olymp", "key_options": null, "keyfile": "/home/henrix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "henrix", "validate_certs": true} 2019-11-25T09:08:46.838 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-11-25T09:08:46.943 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNwmgnIsDDJmsmYzkTAEo3iLj7Jd5Ni2R5vD1Z0NRZ39zJMtbicZ0ec3nAMzFyHrmPnKNFltS1mhehbLbtKNKq1fSo2SMHs11z+N8Nl94F4AY8IhXZ8U9GAorEJFVIxHUNEDqwl55Ov6MfP60UI7rNgy2rUt8aOBmAJVhLEFmJJawJgyPhTrUZAT20OoILphrt81o5bTNfmx++Zs5Dc9VtgIh53hexwKgTvG6fxfKdra9m+J2GXpniOsZjPJq/A0je+t7H3UlxCSu0dRy2Un6l5ZAcoeg74SP50dFgGaxkTQqI6wle6NjtPIdssgrdYTgTcOg9M4jWqAr/5drTUSGh root@centos7-bspark", "key_options": null, "keyfile": "/home/pbs1108/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pbs1108", "validate_certs": true} 2019-11-25T09:08:47.149 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-11-25T09:08:47.274 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDnvVnbK2SNfL8hTmrA09L41RudKW/F7Ty0vvlHjrFjDkdWXVgrafFWGKQZ8dSR6FJ9JLU0X59xiEo4z31ini/Dt2XozFMIr7RdeB3xpiifuUUG5Lgsasd2ezniUoOEukL1a9KkCIxNgmKJuvtlHCt+ZTtyL513yjRICByZbnmBSSpUiett50X10vtvaNBu4LikUJKLc8gP0PHLbaGsU3uOUd655lkFWjWhgExlU8bHWiAZq18Ss63jDlSKFVSk903GKSNcHXMHNPYaXlrYw8NFzejoy/XjxEfmF/Lv3MeAUhHQWx25sh5aDv48mzo8FaIJsL1MrD0QxF1gllobWAfp clacroix@redhat.com", "key_options": null, "keyfile": "/home/clacroix/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "clacroix", "validate_certs": true} 2019-11-25T09:08:47.472 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf1IrqEe8zPM6DWf8Y+WLRU+TwX8Xzki1CQDRwK/w/F7qxAuh9cbX8tCTkZUt0L3pUjGpM1C3bkTzLHaN4bxK1RQ/klB7C3AXjNOJSPJSjcn/N3YuKU3sKWuQkXnQTwqUdTzHuZ8lQWywFItmwQR2Y6T+xG/Ecig+vdU5lwQ479gtt/lB9buI+T10Lefk2bUiYq1PmKSCIF4KtsIdmSazmIQyvsbsBVblMEezCC0auwo7P7SdfSaobf4x4ZZyJfYkVxiL6eGJ8vmBSbJFrcVMkAO07wBtpn/gQa3Dt4xUdXXt7zBizpMmRPLm8m3Pl2kIJhqrIa1cFXPRr5GJ0YXzR mu3e@Mu3e\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNisR+EK2sG/E9dSeoEYCfgzIunFnuu+SFWGyWR3K6Q5Evt7bVAsGnjglNXdVgLbGV4Il+Z+FFgtR/oJgtUZ7G0IwWKG+nlx06hOu2DObsgM7gcIlhrf7SNsI13Bit0v5dQkyLwcmpdwioQyUx5hnx8UamqVo7lLtKsHS8melz7JE/+dnGwIMJLB1Ot/8PIRAb+/NRXpnD/lje5u8idr43bhreuJ6Lell+h8X2vomuvLlA5XjAoROY6L5qKoM3+bD4QXbU006VHg/7wl9tpZIi9XwAuHfaktsmfks+u3eYFCd9nYUKN20zbEecoAA1rOrCF1qhgwTYpa/Rmh7Iy53p mu3e@thinkpad-t510", "key_options": null, "keyfile": "/home/adamyanova/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adamyanova", "validate_certs": true} 2019-11-25T09:08:47.587 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABJQAAAQEAyiGmj+nOAD+il+FvXvZSpUOJQp6hKh6UPaAIQvQZkdeNRWN7wcDXMUlLj+TBFWa1OO+9xqFlnbgVMDR4qy5orUBYlMapUb1rutjbk64XeY58b6uguHwSoMd81QpCCIN4KOk3cldV8UiyX382baoZ9Je2hsOnnBGMsRtHuVHZ90Bj1rIITkwajq//KRs7kjb/qhjk0xkipDIcaSwKjzPlmmnYn69noODwDexpvLYlCF+LowSZEdrOUvVIigSFG7cLTDaU2tLhxSgbvWu8f0pwvjsyW7gOgM+VeRZ6uHC79F1qvgdgQSxflddIgrQpH0rzf6RNxQjR3SBkZ4V1BESa0w== epuertat@epuertat", "key_options": null, "keyfile": "/home/epuertat/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "epuertat", "validate_certs": true} 2019-11-25T09:08:47.794 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-11-25T09:08:47.913 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDRZyWiDO5JqWHJdUUBr8YReiKXxnL87juFAfkcmw2E9KmANVH2sn3K+OgUCQfBs1Wdg6AbcL2GyMm9alodUruzOm127/bH9OcGmu+8SNsMAYrAF9EzsKJa8Ba/I05rl5a87p1jbYG9Rpo7u3OvLx0LjgqydRYcmjwo/niifYNDTaN0K3v8s1mN4TorUZW/i3XBMBIBjlIiNGs/I7IZreR+1N4V8Nwrp2EWxDv7AwG3nvokL2RTHcsV0KgmAnciv8nXXjIf+uqOPuGA+F2qW/wz92bG8R0O4CjH+QZt4k6UoUIXFRW+LdOVAqmUrb4W0OJ8vsVeukW46SvPnYUTkZnF sonea@think", "key_options": null, "keyfile": "/home/tdehler/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tdehler", "validate_certs": true} 2019-11-25T09:08:48.110 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-11-25T09:08:48.228 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEBD7CxJJfbLW4jtuLnTwkxVbhCfh/JL3Qj8oeLEkny1tE7QCeXn1bP5bScWSgLH34Wh4TAuH+8/g/koFwcCy1YuoGBdBV4KV7O9MTnFngdqNY6LKzCipm27tnWZEa10hrF8dRFTmhracd6kmmVGHgvGQPVAVONmix+2IZ83wdj8XJSQHeE3PHFLO39tU+ez/T76QaiBLDVo4ctHVt+N9bBbHV6Cr8fyiUUU6YEKMlcedvxGpnMK8oeOSNuDeh+DldcM9KYzNGEOM34jGRIvJp57dxQDQCtPPW5hvBXg7yMyrv6TcJzDd2LFD6AC/Qw9yrEQPXXyBCDkAlCqDR0+rH", "key_options": null, "keyfile": "/home/laura/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "laura", "validate_certs": true} 2019-11-25T09:08:48.438 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-11-25T09:08:48.573 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDf1IrqEe8zPM6DWf8Y+WLRU+TwX8Xzki1CQDRwK/w/F7qxAuh9cbX8tCTkZUt0L3pUjGpM1C3bkTzLHaN4bxK1RQ/klB7C3AXjNOJSPJSjcn/N3YuKU3sKWuQkXnQTwqUdTzHuZ8lQWywFItmwQR2Y6T+xG/Ecig+vdU5lwQ479gtt/lB9buI+T10Lefk2bUiYq1PmKSCIF4KtsIdmSazmIQyvsbsBVblMEezCC0auwo7P7SdfSaobf4x4ZZyJfYkVxiL6eGJ8vmBSbJFrcVMkAO07wBtpn/gQa3Dt4xUdXXt7zBizpMmRPLm8m3Pl2kIJhqrIa1cFXPRr5GJ0YXzR mu3e@Mu3e\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDNisR+EK2sG/E9dSeoEYCfgzIunFnuu+SFWGyWR3K6Q5Evt7bVAsGnjglNXdVgLbGV4Il+Z+FFgtR/oJgtUZ7G0IwWKG+nlx06hOu2DObsgM7gcIlhrf7SNsI13Bit0v5dQkyLwcmpdwioQyUx5hnx8UamqVo7lLtKsHS8melz7JE/+dnGwIMJLB1Ot/8PIRAb+/NRXpnD/lje5u8idr43bhreuJ6Lell+h8X2vomuvLlA5XjAoROY6L5qKoM3+bD4QXbU006VHg/7wl9tpZIi9XwAuHfaktsmfks+u3eYFCd9nYUKN20zbEecoAA1rOrCF1qhgwTYpa/Rmh7Iy53p mu3e@thinkpad-t510", "key_options": null, "keyfile": "/home/adamyanova/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "adamyanova", "validate_certs": true} 2019-11-25T09:08:48.781 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-11-25T09:08:48.888 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC2k8xsrsLxEqJx3rV6ZZwLH0OjYCDB2uIYzcAyrKEOKlVWO1ICKxkw+gZ9tSiPFNLfq13io9BIrAKJwlrdLCJLmlgCQwpwUQqzmRlRZEm+WmD0dl8LPeQSidyJkPkJgYBEshFPjDHIj10Ei1pJS5svI2l7G1wMWxlt2kHL8Og/4zdFTRctxwqZcvzLmRQcs8lSSpqTkhfeKfmrpR+je8XFbCJJ9m/nJbgyCiZKaGN8nf36pCQDEva4rvaSrK0a6YxL0RWN8W3lqtBPKIehxffr3lYDEkD9EUW5vf6AUKC0ioh4jkPo9WCjaoFsOauMOMVQPHehEAMufEO4Kym6nfrf yaarit@localhost.localdomain", "key_options": null, "keyfile": "/home/yaarit/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yaarit", "validate_certs": true} 2019-11-25T09:08:49.147 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-11-25T09:08:49.242 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDUznqo3VDmcKRL64P7sbjuxXCW/0ljFTc5zZoZ9UvtZvU2jrRQ2X9GyxcrcQnAWlF5YZHqeTDU5o8ixn5fq1MSmaPRBNDKKVQOfR/vFQV73LiKx4BFsC1/EdaPie/XqltaLDTtfqhxADJlz1H3a3ZlkeVHMROTqo/TidL1YUYlRH4qHxWWBuFuVS+EYfTegSZdjwp+DuJA0iN6X99+DfH2DhiHENjEmlbKN13+H+EPzm0K3ya2w8D6ANodB3AxMn2tGs8mLtG/P4kCjoDnoUVFzKTyUpTtgOjis47uSY4dL8mTbIK9kZAiFxnT5sMClu6KRStDjRIrxbTBQJ8wvlat admin1@rajula", "key_options": null, "keyfile": "/home/rpavani1998/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rpavani1998", "validate_certs": true} 2019-11-25T09:08:49.482 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-11-25T09:08:49.577 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC4NYxUHx8HMgbVIHadruN1kAJS5be0aZA9rvGtYDfwD5siKNfXFRyLG3thkjxtEi7DPYMxgbpFOXW9EQKh5sQ0ohsADZOi8FSIAfQlOQ+7HAEp7DjBtjxoqB31jJ1e293YtO+nqAhcOMyGfDx+71w5sS2EFqy7PscA9gD9aV2iMxIR/JD1LxXabiqj12jU5AOxr7ZQ1+gM9aroQuDpNxY4U8jBTxx5OtIYRkeI/aN2gQj1mXGMKGo0ItFn4lsXGLxGxMpa9K2UFw2dWQVTw9VNm79nixrCX+lC2nyaJPsRYZipXF1ID2NRk7oJ79dJ/uRBBXpunihlEC3aF+nOgEPH ridave@redhat.com", "key_options": null, "keyfile": "/home/rishabh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rishabh", "validate_certs": true} 2019-11-25T09:08:49.816 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-11-25T09:08:49.899 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCeAhoTum3/VmRbxSyw5gwgcPF1EWFK5mEPX3PZyo3xPsUAJazALfge3aEuCrZBbsjOCoj3GfFhr/ziEF+OWvhBoPK5hQzP0V8tE9cWJwvHmD72vGE36o1hFO0ONE4DEuu5KRDqQJUgeyQWXsP26uHTwIcvMpy8egZOfyshqF+lQVXzsAex7Ninmidsf/sRNLhzT1kLrKT+ZMSbZ7Rqh46TMlmLD1LzJZNcb6PGVwlfQZmV6T7m0COu/skeSxMcy8hqMvHFNvvfFSqkfMw0KBGLDxMQr4L+e7sXwQDnEZHdzoh0iS3xDp/Onr1FSmPyEjz51e1sMHBsqklHWidSZVLt/+giB4o6Wdgzc0um/QxW2Ne2qjtDlViOgRE9Fzcw/cRu/Xsn8qtXTxYyxXswjkA8qX7WR22OYgjYChw9siUsKkuD2yAbPMVl+cZWDeBhQnPMbDMVaghS4SAVt5GK16jij4+r3IE9vGHBmGwJYJ/OrtcxBrUrDopwsGLcU+WHdv0FXuiw+YbfnzpthC8rVj4oeJ2IUOQ5JB0ub7NmbR4QWjc33JfD+AeCB75x6io07V2riRjMD9gLC5nwytJNG7Yytf6xI2i/CprW7HhsDs2ykOVcDSBMnL1oGW6FeHaDRW39r6BXSt1jjZQ2E6pDEBTDO41tPEL9EVvv9TK87GTylQ==", "key_options": null, "keyfile": "/home/skrah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "skrah", "validate_certs": true} 2019-11-25T09:08:50.148 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-11-25T09:08:50.253 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC9/Io6M3q8yQqoPNPKAPJ6Fzq18DMfmNtxcD8GUWNo0xcMxd7XOlarvYc7INDQjkpzx6s+aqMme55pSUZuN5plW+nuFmRPqBaIxyeCnNjzCch0wbtEZe5dzT270FhzM1Tu4W0dw+WD6zrGjXayLNs2uxNp3UrWtMTBRzd0IfxJ1R2aJtiCE/HNxIZn+bhS/OmEPV1izz23MvKXD+nRaeJK6Aqrg1zbnpoNskz2fZ/AnmhnAMc4EHl/NUyW1mi4g81KcQMwE8poWmw1NpZje9gWFcISVClLvvhk/A6at/v90P7PisX+/ulTA/QYJYvGBjgbNtURYu/PWzYBOAEZQGfj shilpa@dhcp35-184.lab.eng.blr.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDqRSG3i8QY7t9sp6I8vDZ7LVdk+DDrMhI1QmvU2z+YQhUGDBOvvO6YP0gUXPDyalyj0sJXiwLWPmkrOsbnR96YCUcIt7Ba6R0d0w85JxcJ1WyPcsg2UyOBwq4A+s7VAFZm6FEWzlaubixsNp04gKDqBjSjaqA1Kwx8/Tg39byay8Hn3VJeZ1SkRukvhykz2e6gl7ydfuJQVmEyO3pTrd5157ZqgvQRABT10wc3vR2m4eC65hU/v0ci7HMY9oVtGEdWVBapWMuiocrOPCRWF5zrMJwW4vzPOCHa/47hBEIWfOWJidUVzAawnM3nOe7PykvvDK/iP8B6JpxKTE74eLeh shilpa@fedora.lab.eng.blr.redhat.com", "key_options": null, "keyfile": "/home/smanjara/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smanjara", "validate_certs": true} 2019-11-25T09:08:50.473 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-11-25T09:08:50.589 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDpp2NWr5T6s+jzHiz/FfCKzN7ZBnUSzyE7ieuntzjeGotyupAAjVBoSpCJgNDybm+WkE4dShv5mttPJ2/q1UZCLRj3nN+y3NG333aLZuyugR0ht5qtvGVegg58PwQ8CMobR7ca7bl5UdEy0UIrM3d/qpDOuPkuE4MIJcrtZGICE8NAVnmHNOB9Sm4ovAYkewTuMAv40j2Dl4F9dD6bg+OJfBOJ9+Zj+MpowYlAILtEB9Dc20zB8GGUEtr9yfOVroblTYhvTxGYRaE7aVBe83DNwkWGbDc/qUNR6bpA04w/P+LclYVsVKRoHOs1EXHSwNMFromA8RsKS+rajwu5uRg5 ben@dhcp-185-182.bos.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC93wGgyyQmTxa7f+BZcYPUapTH2N1mnV1ikl9nzJzHhgrtJ6pv00LhtH1HqGhvoqnqZjBD5XRAKyjMRcMBT5MYcXXep5RqZ9sJ6Xz8rHkBjfl/OOQUeZ/ZyG/Ksr1jpGraN2xtrjQQzbqN1YvirCqfLk2exPvVeEk4CGnNybwKwCWob1uKV46ASrYeQK8pfHxrW7pTo+xdErHOvb7uumZm89/YisUJGicO1qSHBeuw1MxjhnD+fIcxHKWY5NNJ4qScW6a5AABk1JJsoHkh9IGCVaf/zJv8D3gil5SiM8LIBKaNa1fn6oObPl/Ky/wkiIo2gmUZawvAVA8iJnX9eUPl ben@bene-laptop", "key_options": null, "keyfile": "/home/bengland/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "bengland", "validate_certs": true} 2019-11-25T09:08:50.798 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-11-25T09:08:50.904 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC3YhI4am/T6s4rJRvbC/jAvrNcDhE/zaJiVDAjEuA1W/5Xt9ObX52eAnp+tOOtelXrX8ylYF/z5wGNQLjYrWepXhuqaEGxeVayCHzts6LMpmvyZ6sYnAw9HFO9rzrTawm70DuvWqY2Zt4KA6fM+lfXk3Qdpi5RCPDKk1jpXwSCyUtUKSWTDRtYtvvJTcO3H/xCJxQq/F9sU8yqgQcqFPfMNrV4a8gNj9KACqjt4IEIfuxKlKhlMA4yu1VTmGcNuhQS+geJzsEGuxZUQb0Io+q4o0b+40BQtazKgOdTcQbgNtqh3wZpjJsvB4AM+GNPjZY3mCsd+WPGg6dqfU08Voo2b7BfSwyCn14fQHnj1nSwRIea2juFY6AMfVJjstM5iIz7lOH2c3ydWWhzvSmMeoQ47zBXlg8JytWmDrXiyd21BSIpTL0Tn8P95wHYka8U9KRV0ZY7Jr2w/ThPdy2C+Cdir+P+OEL1sKHSnmEyQEeONa9P5yH1Rd1kM/FNESAIRiMLGspzXQeWoJhHswVaKoSBW/7f4b9VENxda8qAQi6ob8A71cAjbpRrkUmd5EBCSFMNs7lscus0/deUVvlpsrMjDAJa8S9gqhGHNU0YvF1WBIz3paLRAHDm9DWrHc5lreizJzX5ul4P/Z4ITjpO+9ZxKOcyi8wnJ8Og+QUYLpqOIw==", "key_options": null, "keyfile": "/home/pnawracay/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "pnawracay", "validate_certs": true} 2019-11-25T09:08:51.123 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-11-25T09:08:51.242 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC709gIxs1RgiA95ddp2uymzgV84VHJVVMj71FyTlz6pa0gtlaxF+r16p8dnNWjhyJRUXMwBKFOwHx95bcdfiRA+BfC04UnI8VvBxoED5Nt9EbKD5h78ZoNEeQJKoG1dsM/kWWlTd9WCsCyzjRz32+YNWul0LOhrfloHP+LHnfjf/uOnzd8iRHgnpWr5Z9uB0MBu4y/L6AmHnR+8pMZQq6Bfno8ZLwNCzx9v+4MqOz3jY7FYv7SJIftQH0fJygIfCz3bKyUEWk9a/JJacETuJGBnHgNAXUr/FtM1IFiMl2SsTixRgnosEz4pX8EIe5LJyw/Qudro8BPEC8aIGy0XAkerGmTt5iALMnzmeeanZCFm4rF/YiFoHLMS/B0uv42MD64hB3knrV8Bcq+mRjwVOYgecO+xoiBsU6FZeXaBk5IsjmYTwataGv3ILEmNpss0WeqHxF9nd7oTLnw51cZfcaerOnJ5IZJZTXwLNVr7K176BunHggAhjvtvwdeKhphqt9f3ilLr6gzksEoAR4JBOYJJtd8sEqHwBtti+YbMGwH9Y+Z+SujlQBQIsom60xj5OzrwjsIi5JAtOhbc1Mq2fptSXRsR8ZBT8aUFqZ8R7ixEqq2F47OAznbhSQKidzwUNkWNI3WAKi38Oi4aya1k3iqtL04bGf8R/xD5Vq/2J+UlQ== vant alfonsomthd", "key_options": null, "keyfile": "/home/alfonsomthd/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "alfonsomthd", "validate_certs": true} 2019-11-25T09:08:51.439 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-11-25T09:08:51.575 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCoL9+aMxLtUG3aRIuT4kbjc1CPuQ+wbS30ocuJ2YIExlKMSxVr+LYqbjcz0T4f8gAKG539dZQgD9BAsHM8yfOVEvei0KsQHn87pzgYGR3DtaMx5IEG5xNyARPu+VuEoe8qlL3cAPwTiaU78datD5Cz9GEClhL1Fhn/Z8o7BMbs5xlcUIQ/J+s9/sKMcqmmBJy5IHQIEUHCl4SFwsev5g5ADlnLNNxjpalExuBCEeoSma9F3b2YEJHvpA5JhCIMZpVfXbgh2UhUsyfAQN6KgOIQ/XlKBMQDtizf2vZ4b8hW4ELcvKSADfTsyyqEs/Ql8jEJu32GFKf2mYPbgTS6E7FZfQz77X+5OBhojFAstB8LSSrH80FrG1gCbVZLnW0jXcSehKnU0oNmUWOKxxaT1b00fo/iZ4P8O1uJRY17/uaVCPNKdGTZzIlzyC16wsFFBxBl0d7HfojQY6qfww7AQJhvDkV2sP0/MIJzwriWOzZEmfr8Osqery5vFiYcSvMbdMZDfMEYd3z7Va2RiTIeyXuEYp5hzf8eyJRP/f2FEw8/BBrm5JCxeSWUx7qwgPHrJoWAVxCIg/M3YS/vfVSFc8Hh88sgc2upq0OqYr0uq6EFzUZi4aoOkOVYn9u/uc/p1AR0KbiQOkERMUDWTyADxrHaL0nQmGy+XyiJQjhq5ckZ+w== doliveira@suse.com", "key_options": null, "keyfile": "/home/oliveiradan/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "oliveiradan", "validate_certs": true} 2019-11-25T09:08:51.760 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-11-25T09:08:51.938 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDh9bzZJulXGES+l9Xh6Aq15RZ8uQCGuDNhlNQtDblE1ISKJ/DqGYXp6wUW54+oVNA7eZiXz+fi3mq5pPEtZZOfd3ixEDzDJ4E3cVXoDZCqeWEmea6KvybeY10YxvB56TEI8U2KKAd56PRl20klXpLCjzXNqG7n0aXFcpCMbXDu369VX6lOk24K/7++7Wc5SttcvVu19sT2kqzsB/S1Y5cxiE6RM6wVtqBoksp/kRCIA16ruNwx3GUabDfbEoUGXlkkUP7+TZgAbHtBsYy6mCQhwi0S2+WG+HhHDUPjHhV+MdN9ffibCtOEGo52itVLVky09VeBocuA6H22JDGXPBgjcgf2NsZIqcKqGHhUkXmH92fhRSFOBLKHstrBq8jWRP/mNrgj8cQksDsakQYQbDg5dyabp+M0/iL2Q3YVq7erZI8aZMA7ZF3WgoQNYZg5E7oejM8URIlFP3x1ne2ClRC9a74phSCxeU/NVamGN3G3dImzEXGOSNyRggHJ4jGrIGc7tLPCzmI5OkomcB5OxReqf0r1TNXuUAqw8M4EoWt+0xoAmH5zVlUHf+psUCJIEV/4pbgtoJiSNq+LVY4jyDEFbvTAL7MqyXorMV7Tqlj+/3d7RXfhW9lR/SHb1Z3jcHvz4ZzYMzKWeJEjz+Y0NwIdFDhcPmUOYtEDmuRRrgYiSQ== sebstian.wagner@it-novum.com", "key_options": null, "keyfile": "/home/swagner/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "swagner", "validate_certs": true} 2019-11-25T09:08:52.068 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-11-25T09:08:52.281 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDzYzqIDimPjMOUOUz1A122eV52fl3mgGgNBRYRlOWl/lOxM5oOat5FPkJlnXTVyQFl0MlnYgIv9fTnjHVTM0wFmFUItlUl151ZmK5XExiGpGVsqG3UWNyKJxZpk9Uh2a0T/5CKzzwXnv2fJMNxzfqgopYDrgq4TFEZcdvgWJ39dcRTXST/rGJQ8wOq9z7UNzKU5VOGxvgi6f/nGevGIHrSaTcY3Uu8BW8kXUrf9vfl+ZSxVjuNHXF/VmrJH5Ne8mVu7DzPMrl8ZZjxsfJKT/eQqF9kjqht5586OSPKIwg/LEOyYj7Wyyf3QbDAIp5bCygQfmBBl+kIxphJYq3EwUZT ylifshit@dhcp-4-102.tlv.redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDFgwQYD9amVSYq6JzjHyGmKBJATQCM9xsf6QnF3yb0Lvry2nzY9F61evVXQ95TmF/AqasRx5p2WansIvegG5ag5mytvRwFXtHzPs4o/J+XjA9o4+A5l6j9cOFLIDD/TwrwlUjgMa2PhqUztSpI1op906391RLrJAyflfDghJxeVl6dJab6TEsslV0W+yaCS3BBxMXXrCbxCB5Ms3KCRKDz0DqbhFc+Nw05L1O1pC98HBFRIp1SaXlMcT3++CRQ3wugpOMkDMNLbycl+vdMBFY3UZ28rRowS3Ta7V5EARZVVH7KxU5cL+8Hr/8DE1/ITOmrluaEZ1KwYgoxZbZ3w7Ptw0FK6aiCZ49YtqKT1WakoVfO8m7pCdxkqUOA91loMQtxvCwZ/Vfoxaut16gklmPz8N11xvo1H5XB8/ZrvK5yKdl3JDl+wwtfE/fwo4E/fte+rPRisoWBRsmsx8EGxs75sXnb6n3VjZ65vYEKsNMLg+5EAaevWLLeqF0a42UWGJrHdb4VKX/OAplHrOk7kA/VwArow8f9n+iaRp2hgDU9OhQiFVRhgqwahlmCFFXkcP4/4WlxMCuZrsaNMZgMjVhfsp7cskKOezSfZZey2MB2fREMS1wg0j40CdYRn4jTNEBr/LVszsoM7t7AwXUpnqXeYIe+gePx1PJb5XSzSPZ9Pw== yuvalif@yahoo.com", "key_options": null, "keyfile": "/home/yuvalif/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "yuvalif", "validate_certs": true} 2019-11-25T09:08:52.416 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-11-25T09:08:52.625 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDXa9uTf3q+7tGnHzSxv0WbgpRKrBIYjAZ5x3V74vQUSD1aAbmyvOqgPnts7XttLYImjjRqiisyC03L3KgR4b3SOeyL3KFxrk3KkHAJ/AJH+Y/UafyDhHNl3gRNg0B9rjsa7h2dSlcrFKnAX0pDY1HapuNPrzYWrXsEbo7C6wteFh7PyPmTkHu0NzIIPC4FcLl8SGaww/HeBh8v91IymgFSgXG0jNC063FEExWdti54yNLGh3gpW0FytsiRN9eEOKUOgcveT1tejz9ORv8evO4Y9SCQ/lsyOKQKNcG5C7J8nsbwSjqQ2cf6i2aYFZ0Oq1UXuhJ8APJCoiKgLJy7AiYf kkeithle@dhcp-100-19-89.bos.redhat.com", "key_options": null, "keyfile": "/home/kkeithle/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kkeithle", "validate_certs": true} 2019-11-25T09:08:52.772 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-11-25T09:08:52.773 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-11-25T09:08:52.954 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "key": "ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIBYa5uEOlFv1s9K9D1SbcaApLwX3LBwASSnRXDHI7MtU paul.emmerich@croit.io", "key_options": null, "keyfile": "/home/emmericp/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "emmericp", "validate_certs": true} 2019-11-25T09:08:53.112 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-11-25T09:08:53.114 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-11-25T09:08:53.293 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCj28V9sMybkAzhkiaNiDDr8TfNVyr5DNdbOqszdGEVLy1Rxh/UYnxUclsR9n8lt9VnCft3M4NDTnVwlxOHE8+0oolJE9HP+DQomsgj3abP04EGJyOA0X8E7eIFQNLRbMFCD0yX/SNNlYPsS5fPTFhIa/gbBPmDBE07kULts66Tvc0Yo/2yIfwV1DV34bcWWVGi01XybIHBl9qbvlfBKAUrFEcv3ZdpHNhuZw6IkZY8xoYxo8iF+toLYeGX+gFxqCZUUOO+NBVmb0hVL1LI6bU66IrMOQDkFurqpvoraBTJqwZgXvMPceFUkcVIH4Nm7cK1X2WyMtnSw2XBpLMT8C4V mchangir@indraprastha", "key_options": null, "keyfile": "/home/mchangir/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "mchangir", "validate_certs": true} 2019-11-25T09:08:53.440 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEJa/U9zUfZ7YFl0kwU7Mx3rHPn2O4sdh1J3XBUgoVdrJ4Qnh7i+QhN+fw0eaBWQE2GIsGEX6YWAL5xJ7EuYZyas8OVLNdP9cJIr79M51d8dG/ywqBpgcFnx7qhCSua3sKmP6GCTWGD9Y0eWL2Z6y9t3amb8dPXviMNqKGDz/DaDtp5bPaoG8mDHQ9QeADYYk3tx6V7+phIBXrI2GKt0BBlJeuZysiUI/XWO46JdMyHvCzvzcdquD446aoWS44waDGj501sgnHVQ7gwf4P1+vFOSEHzkWhbenpJmmTA26f4dim+Jo5JpBW/Ma5mMBvORFhH/Kt+S9P9sPCiF1shWc3 d@d", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-11-25T09:08:53.629 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC1OVCo0wMITAp8PNUqsTRPFz0SuBL0r1RzKFJUoxQm7gsP9qttZUQRlfiuZ7YRzkuGLOEBngPuIcV995o0lZ0+LVHA8jxWEJamb6sM9YgnonbGKDHfl4lT8LJd4YtjcWnMGvUUdPiPgIkfrpxXtx+RzJ7eCTmgRjAJziZrRh3AqXuKty+s9MDnmK8EQ2smpLdkMtZG5W3U2PiVaGgq2FIIOAoqr58xGqFgZ4tIpvWFpNxVsDZXIg2NdC+ouuKQvuJRWq9eERRHwq26CR+kDFNZeYSyhYKxNBqauLCsAoXNFCJzrdX5Oi9Ox4lJTsEtYX+SMziVipQS54F7MnRAUccukQ86IbXz8C5Ok2EzNo6smwRZ6UPKRvebWjIvfbeR/iwPRnrWcNgjGcaTR6yhlRJh8pBkW84nyY/bJa4WXsoNLgutdsYkjl18KKhbBldjdf0ovfL7FVgqYfAo/wpQYJJeMebP5QAczxvZIegjHRrAvbs/nbQTmuAkes3WL1hhBjSTwyIbWCzAuBJk8MNTmeZkVBvub7XoIyIySppfoJC/1a+ZsxGkzAlgrIxXPYrrBzTtXd/9YRiIgCYx3UtfdbPkE6hT6GblH6ST6QESfWyyvfGxwlOmu+F+icRyVVJbgM9T/PD2W6I04u2/1mcfoHwaQGdwvbbBkbjWzm+zwNSisQ== sanupkri@redhat.com", "key_options": null, "keyfile": "/home/sidharthanup/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sidharthanup", "validate_certs": true} 2019-11-25T09:08:53.783 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-11-25T09:08:53.980 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCkeYXWjU7wVdODC32U40QRFgFH7ZEuEsOeWWaKx3Y5EBuAcwkTHiF9Bq/2mrDu+c9W46S8L38Ogc9GZ0DcCXtN/Z4z8iKOIIi5U1GzvwdbKuiAPCd8bdYP0vo0MOIRgylJo5hcnB5BCwCXRN1hmci1ES5UbSeQg0rydMiyYXHZ64AT47+qVUpn/mVP6PVw00pPqt16l9YxhbPFG74Y/WnY4FyrqXGQJfv0tcP7+UR7GQdFiI4QQV3ZUSRHRLWGcmoiiOmGimLhtzEmP4Jzy4gF4wg9q8jyX54NjfxDRZs+YuaoqqWKz/HSz5ic7W2RGKHrRby2N9wNu19nmC1NzIn3 varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQCW58OUR3pniaYgvr9xAExeNm4eMxgaJfFbEfEYPztTWv/xxqqihBYCeFltfgWbu7ERTy+hVNo1PJWyyczjl5Ivc3FdRmhykosygs7X74PPh02c7Em4MsSTr7SSPNN20rkJbWqiYMMOBbqbiZ7JXiSUNVilLHgZqZBEQc5Rf5I1PyBLHHJ2ZruPKnDPHCRLdwJ+MdXy2haKmGrQLhtzIKaDzh8VkfojL454k0aHCIh34Z0oBdP9757mNuYJUVEkKZF5t2oML84lWGMgwZxUWN2yO9ALTti4tNH1fXA6NipS9ztE+3hEJe1fWXEoUQDX9JNu4qEHTeCz0secQMkyZVDZSQk8yDUQxQ76yiett37sQB1WQR/3fclcahmqQ4dUS7M3Onyz6k6zia896PBK6G48TBUbFa1rw5Hx7WfNZluDMkv4YxkQvfz7XVzZx2/lhs45Kjh1UR9n7LaW/NID98vokcvASw1N6Bk/6pOd/45iPyEJemwovnQLs6ecwlMIikM= varsha@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDcQ8xMjwVf+KiFIQARcbFT+wg4NHysBRrFxk3Bx5X2k2hGKcEYR2tY/ApHZYfJ6me3LUvJEXg2c/6xa6ibVTUp/90l4y0cF6RmF5uB45J0xuBYoFJd8xJkVrhDIlZix4aFfb7TrmJEqjleBrkTwam7Q9aE3lss16nVOxU4hiViYiCANJ45wobK6tz7y5btfzrHE/iYpwniDvzLwK7EpkEK336VVd6f9wV0vG+ErEFxnNVxheXLLm05LuvdVQvFqJcTFrc7/GZ54yKg5IgwVwXqSAyOXIhgduUqk+oN1ciA7jnSqdiOKQPS/VmYkbhw2YP85l8wGVreIYUmOG19QEyVLB82oB9KS1LezM2KCH61XW22+85dHf4PvuL9DULhGt6wBCvpO10OHxw0JvHXbEgBPhiyd9j2yJmcQozbRPth/sMUvLKNXOGabZhhkPBYL7xZICHc+UwxrXs0h/j9jOjLyHzBAWMomF2lzgbvtzjcbxJ1a5oclJ6IW8yHJllkOh0= varsha@localhost.lo 2019-11-25T09:08:53.982 INFO:teuthology.task.ansible.out:caldomain", "key_options": null, "keyfile": "/home/varsha/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "varsha", "validate_certs": true} 2019-11-25T09:08:54.119 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-11-25T09:08:54.329 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6RDgqwA0GrvE6KqRaGCgYnFhnxdi9ehjIWvfaMa1oq0CqYxUAee2bNbp/X2Bd18Vqk8iH9fpE0+qrLgAH/8BpD+MtgGikCFMORvazeUtOCF/OCDO0AMg1jvl8aTdvrQU8ZTDPXa65Avpkc9vwugcoplnwrW7lKx++XqCWsSlfHgEdyMV3wrt/ZDFwI2AS2bZBpWM6ViG1KSjXjgTi/T3hSZPY+buYvWjzoEjQO0946iI3iiKKCGohEpCUnG02oJXcS0vvgl8dWE71451lbainEKJ4hbDhfDmy57s6VXfVUANsczwiPWiSC/ZmMlDaCCBeKHO0Gz0xXaDXd9j4SA7P sam@Pondermatic\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCYylAUS12hF6XndDQ3CkrUddKmXNSd8pOlHwO+pWaDTJPKxwtPL6sM3aOtaAPLrGkv1zD2k3iZaxmDwfPYGaXOyb0KS5ZCddtOyNXj5JLrjxh2niqFPP8SIkdHLOP+0Hv1/tB0Jz75RV+tNIT1SJTm7OZ8wZNy0qYycITSooJ14iEgyZ7rOa3EEzbtIVpHS4+YqIuIJdV2oV5egCrJyRv4sPUrS8ERXD4QZSOlHE+W4Tf+WVrEfsHOdDLRlAXJUzhuqgWtVouiYcW5ex294z28eA+12lKaPQpE0cft/NJ6FLs41yqh4ZN588bl8Ada+yTXEaHWioQEEJMlca2jIgrP sjust@rex004.front.sepia.ceph.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC08RrDQ2M7Zjcpx7RL9e+tjv48kBgDNyTenX1QdxW22JA2JR3D3ThA/ej9HtiPWcIVz16oZ4tzrCEVwaCMfb 2019-11-25T09:08:54.332 INFO:teuthology.task.ansible.out:smXq2z08ZXYqwiBuoXJVshU7lIKVqAtadT4F6Q9eQvxEo4zWdBib1W0/cqNLlYzU/DuwlUqYnWBDLGxc4fD5Kxrz366oWUDB/ULCpl77TMqfTKlStCgGXot0zLqiscqzq1W7XsuA7A5vQw/JTq7MvIME+SkUuGWX0WKwTMIRZKNWBGmlf51bNru3GmmAnKBUFsJv9H4G0FPqGY/aH02rr5dAgTTmfKt+XbWRfMTMpjJqVRGX03+1mQNoYQ3kNDz0wl sjust@office\nssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA5UMPySf/h9r2/OryRXrO9Qq9k5J9Dn9nvJYyaYCE82H/qkBJGboAJgyIXfbsGYO5uPM25d3fGRH4magzAQm2hGWNbbt1YWn9HaIyKCCXo9JX600/dv3OZbHyGGmgXPa4Anl16j1c2vxRRt2+ArjlyyhP7cAcRkilUGdagK9u6ALrwnz9hhXYV+BmXrxMm7zZCmdWHcWz+YxoA65es4+w+aLvVZWmUCZLF8uZrcpPSw+oK3SfZE8JPRc5vWulXbWEEfIoHb9AszPIpSyzbfPWXelmc3XlG0KD66t7w+AnNlQPt3JVePRCfFHPYeTj9B0k8sxPyUqTQIl5pu6GppRMOQ== sam@Deepthought", "key_options": null, "keyfile": "/home/sjust/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sjust", "validate_certs": true} 2019-11-25T09:08:54.476 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-11-25T09:08:54.716 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEJa/U9zUfZ7YFl0kwU7Mx3rHPn2O4sdh1J3XBUgoVdrJ4Qnh7i+QhN+fw0eaBWQE2GIsGEX6YWAL5xJ7EuYZyas8OVLNdP9cJIr79M51d8dG/ywqBpgcFnx7qhCSua3sKmP6GCTWGD9Y0eWL2Z6y9t3amb8dPXviMNqKGDz/DaDtp5bPaoG8mDHQ9QeADYYk3tx6V7+phIBXrI2GKt0BBlJeuZysiUI/XWO46JdMyHvCzvzcdquD446aoWS44waDGj501sgnHVQ7gwf4P1+vFOSEHzkWhbenpJmmTA26f4dim+Jo5JpBW/Ma5mMBvORFhH/Kt+S9P9sPCiF1shWc3 d@d", "key_options": null, "keyfile": "/home/ideepika/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ideepika", "validate_certs": true} 2019-11-25T09:08:54.848 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-11-25T09:08:55.060 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDC0h+TpaUkm2V6Hbf7aPCTys/nsm51CmY9s+bdwEXAiSAWpq/7Xi2LlN0mZgsrAYBHbr5L+Pz8x0YdigvWysUK6U9XRSPM28cbWmh8F25re9HEk8ARESkMZKTv/hKFs1v7kTHbT9hNNynbQGQOqEaazjr0shcli+Byvop27riX3QDDSsd5xfCjjyDz9fiA9YESlF0kbXyFUKPsoaBhRv9FUhtH6DxlygxtxTsKyKgeoiKw2sCfeUMckcJKnGCMtkXW4U8h7NL/fPZFyBaMDRi/TtoBKaVGLaWqF7RuukfjoMCwhuIDmuTG3WhanH4HJrfNenPZkpZUBhzWwZcleao8SD3mXXfZ9gcZb8QB2ymEtTqrAEyDoNWO4q8ljNP3WJXIgF0h0Sjmx/CV/i0ArEYlBMwBD7DFVnPJqCqSyypuQdDGsDNQZxaRCiYg7nIslFOpDr+0PLp78m+CHyYbSFLTvVVJoJ1k0YhMnDPHM29nt7fMGx6Qrigwnuk0yKg1bfgJ/t0TOqEw0bqGKIvHnyoP/n0ObazEsiPl5pRr2ehs/Rqf35Iwhc1i02Hhz/3Sqwc1eAgqm4/OVghxqUJGpRocua5Gf/vO32wwoyGxh2sl7y6K2pNDA3glhKYbZeIh1fjnaTjikgleqIQkCBxbTeAE7n5+T6TcFtnII6RSxYv3tQ== gal.salomon@gmail.com", "key_options": null, "keyfile": "/home/gsalomon/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "gsalomon", "validate_certs": true} 2019-11-25T09:08:55.208 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-11-25T09:08:55.388 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC1q3k4RvLLCaG874GDJjja4eYqFwrMjsTU+aRzambaVuCmAJT3KQMGuwMjTuCIN7R1Fqr+/dxUClamsfsGqPMe0ASHT9X/0AyNgEkr6na1Fvk9NAwAFelwtiRIHhclzYf6KY8QHZN/eBy43prVKDXd30b15gwQyVqU0/xmny0GrYCN24k3N/R3divt2GVqptYgEwS2KEyFT5EMnJWjdIVG9mBOHrgec2VF/rc6w3BZ0s56iKdpOeCgk5ISYTcom+Xrf+fSDHirlviZv05tBP8ESmampUT/EnwF8Mdaln5XJvZPhrq2JpUisi9fVeR+zlzg6gRGubvokdA3AERrhCQ9 skoduri@rgw_vm", "key_options": null, "keyfile": "/home/soumyakoduri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "soumyakoduri", "validate_certs": true} 2019-11-25T09:08:55.566 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-11-25T09:08:55.751 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDPeuSTD7kv8yQX6dW+7Vpq1mqrcr8JGubPYa1DpGTV/gE50d5L42UspUN2FXmICDMiV1h4Q9WQAFma3168f1I5I94qFkV+/56szCMoILClm7IW/EV4t8fTxoyOpEa9l4qSU3qgJpDbY04CkiyVJFuQj24Zq3gD/70jYgGpy8RfA0BihSR6F+A0WZLh/fESjqfor3/pFWhUCRnPEqrZsXyuB8lRgDTyGp4s1cHyUqmFMuCPDRMtok0rSVpaRZj2Zj8lboah19UI/L89ptQgEdqsOKzlkxiT+1GdyIY+jpPCM0a3yZz75l3Qiu0k6q7qUV8T0hm1GZiBguvST1KMcwnt kyrylo.shatskyy@suse.com", "key_options": null, "keyfile": "/home/kyr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "kyr", "validate_certs": true} 2019-11-25T09:08:55.889 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-11-25T09:08:56.081 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDeERrQ0NVXhuTYzI08rlKRbHZVFXSv1YC7sGq6aCWRbTCv7Ad/t/3OJz3vfcd2TjBMvABfrYCFgTFWIaJBaeA3BXUUU9y9TwnzuxKv4C46WJYoalVGyLEpWpcGbJ5v+leA6Je/+lunenplb7TBKGE/YTgggEXyX7KnunYlv+Jdm33TmAZTcHrLk5QR/6FdRGKmdlLna0dXZc3v04+AI8W+p9ghAv1+e8ENxJDTF8p0MawBiKWcSj2mysOhQ2Cj91qZ0qTnLLeu0CPf4rBO+qjYIshiCuTLhYt2UazlzME4nCCsVqK7Nz9olA/lgvJLRaPxgQQ7iCMmIZilS6B4Y3oL sseshasa@localhost.localdomain\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCo0zniUEXuQ4JJuuRtfnEK8WHqr97AmXEniGPY5U23MTxhMuP8CSkGWkUHdmXw0H1tDKbYgk97M6TspJropNGKljla2H1/3e2T1BOI1L2WfcsOh6VPYeyOj2ssMJeEDdQgd0J3g+4oj2QS+guPlT24UhO0CBhRZ6waVlJMgagDYyDLCLGI4ASoBYx0AOaoyjnEQtRd/gyejzv7jntkHc9YNBPsJ37X6hTYNA/s2ORJE7QMXd6xI60mTBuCdD3JcPYoUSvyCUa7rIrDc70lAcW5XtxewvMBogTZJLoFG/d6Ugij49h4AfwrtoUCtLyJOWJX6PTfomXely8Ym6KpN/3981+m70dhcHPDJ66QCWMd1LAgftVXTIsebA+Bxfg4Vsbrv7j1lJXzmMfsOl+1wVPe+Y9h28GyeIlalW3laURZGV7dhexGZ8YNxPuzxtAlWtfw+kddbjw0j4ODtgVXrj4SDxqmTTYYuMBkrQ3H33sOgPLx8DyA6e4pckb6lOOEvb41Yi5tyP7BIJlYN2PpNVDkSQd/t5Q0di4Ct+3vqLRCkPuJiJAUyaBGT4HM7yQc25+Y0B3sXNON7BjeaBXB7OwKZMp9w1jwbJt4JL/aGOw3HVljVyVgu+pIuQ2ISd9VBcmAJK5EGdWe5Q57h5fJkD+TVrXgeNTaufdclh5vWOH1qQ== sseshasa@teuthology", "key_options": null, "keyfile": "/home/sseshasa/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sseshasa", "validate_certs": true} 2019-11-25T09:08:56.215 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-11-25T09:08:56.428 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDIC6uqbIg9XaTlDZd7KCX7FXSzfGFMEWXhx42ktz3ehOPLviA/yNpVd+LmYHSgzd5Szd3mMxleisiE+JdSq/q5YyPAF6eD6+wfynkpMv5aWNHGI0yVfKbCwdOX8bY6yqhyVwMDo14uoFvdDcM4Lj+vz8F75VJHUU5ihJNOW24EllobTLzR4XI6AcM+mjbXRnBLLVoBSuDaU2D0kWqMn8rDCJKdUJnoiy9oxIok8eMLfGkGsL55oOx9BflMSz/0jM9X5GLQahYiECbXjvCIarCkUFE8txNoXM9Djmg8QqhAQDhY0hzJTauFJJaD8Ax1uFQlXONed7tv7a7Tb3dH+/jb rfriedma@dhcp-4-239.tlv.redhat.com", "key_options": null, "keyfile": "/home/rfriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rfriedma", "validate_certs": true} 2019-11-25T09:08:56.534 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-11-25T09:08:56.795 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQDIXHA8BwV4OnqlaHHyYofsp5X+wanose+aneJVx2Or05PcWA9c7xFEOerkvEn9LvivSRd9G9o/S8SvnGH91tG2bbOAidVZ69FGul9OKa/BCJYUP0vVBX09/Td3n7+R2lVlvu5XiyC5SUzCjaQEZLOpjq23rqaqdCDXE6wFnxfpkmcsccbG/xUiOEBeBGI92lntdx99Sn6CxWWA27lMMaYkCRvX6ziRZEm6O44N2nMm+qYm0jmMyGv1iyQkJxtt0YaLPp+X9t2trUEzEjgBCXqqzEzBjB7T4rbzhAuvU3NRHiAoWXSnrY5AQOTTvC/wpaZV/gF2NDNThjMcpcEdNstleOctXtN4XTTJQewbG+t7I9knR00TPpsoUvUtpbcoFOluGgWTDTFa6GCmDPppx23nZgulOniZUVTI2CbScSpc8sfp1JE7Q6vLECY9lJ+GpVCbDAbIEE9WHXSkJ/6L9yf+ZoHbpqCkEFDcWLfz3B5NFoJ5gne8B8uzd6ptKVtJrP8pMvoOyydyvQlmkgh8tNDsOnp6kaUNG0NHmtfGlCTkbM7oKsKbf87Cv1HgYuZl/HP7go5sOX02LfBfbvWrawZzOaZ3dewEq74w5hVMKEHrmJPstIpcHSxZpq33f2j4Z/uKW9tJat6rgSWK1I+T3tCd++76apxbOYl5TDOaoxCyjQ== lars@marowsky-bree.de", "key_options": null, "keyfile": "/home/lmb/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "lmb", "validate_certs": true} 2019-11-25T09:08:56.858 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-11-25T09:08:57.108 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABgQDMDUcToIhs8tiIozgPudiYyD2upEX+VvsiJRti7zrA/1HPIDARYt2+xyWCASm5x5vdohRM7wnBLUlbBVnvr6ZUQvfShNDsS1Uu7hEfYF38MtW+dVwvGlm14J4m1gbjzqVlZvO6bg6rfGo7Qer0bKUlxtqbisQ/kCg/lVWR04CNxdRtZxEpaQ21y4C5jAlrpFTzxQSuiCLEyY3EAKdZUaZElFEcXlna0ysCP8inlzhF402PkSdl9DkkPpg3IR2dUEGqsblWSdLVk19EFavjtGAAnsvaBWWm2rCLNC5RmhGQaS/Z52Aes1nBuT4cI6qLjbEMqAzLwxiOR7l4IPMpK5EcuRK+FaqQG0wMAjxPc95CmxDzi58ZRJ5ERV8vgps73F1AJXWU9Lb1P77BTsLXqiwko2mvPf6e0LwCxQuxYJWc+xvO7Ji011B+nm5eLCCC/LcDtTPjkimTh7nkvUNi/yTKCXi9KwLISAHHd335KnfrL28rU3GahMXs4NjOxuN8eKM= ksirivad@ksirivad", "key_options": null, "keyfile": "/home/ksirivad/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ksirivad", "validate_certs": true} 2019-11-25T09:08:57.182 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-11-25T09:08:57.403 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC0POsA+0NyxecLo3Y821G6W3+jJPKFMuLSIl4FSLi25vuyuM5+mFGmCNWv8gg93eQhG9Y3SOygqmkaxzxDtBS/V6VNC2f8hZsVw9B+HbnfggvCNQSgjJ5rhvf6IHfx/aHq4IHF+stqdcoWhwR/68wIhGNqtP7Nhqnzk2aN1kG9bU2iVk4CH8XGIwqwET3aWNuCMcHi2AhrvDBpZu7bzo/6ARUJDqEfTEC/XIkzv8/mEkJ5igcycARVy6dChuCwfZc8rhfwQpekwx1Y9/WBFdNQlYkN4fVe8yIqT5VsQ3AcRv21GJk8L7mwB4hF9dVAi4p1cSyxatR3kDifsFc8VQk9 toor@localhost.localdomain", "key_options": null, "keyfile": "/home/ofriedma/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ofriedma", "validate_certs": true} 2019-11-25T09:08:57.488 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-11-25T09:08:57.753 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDc/8sWrMq+bqKBPmDPplUgdMoY2SQL+57jRSh27PAoYW9rNVR/UgK7EA9oxFLfoi9wJ9+VKU4opeoohOXLWQXXatLvfXjGE3fRYIonQfwJqKrasqQJTGWzRZ9t1/2Cb0VHkoAvipCJzWmgB46X6q73XW5Td1H3b+AINd6E4Rsm1QjBOhk0DMjCdfwrVUZIfGdsEo+5u/AwioHwH+NDJvXTWT8ZigeZBNHJSY/EQn1+15ut0QB2rvXkJasvvDlVbEsPQtM4C3VX1ronmAnKMvCp1O4EtAXaoKSOx7XEN++UAvVn7czDCWk/UFgpzZOqTcQJwg82XtS4sZP23CpNe0G9 h4iresis@fedora", "key_options": null, "keyfile": "/home/h4iresis/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "h4iresis", "validate_certs": true} 2019-11-25T09:08:57.870 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} 2019-11-25T09:08:58.083 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCqhWVTosH12864VGY5S3/k0yKc59GbV0m2WxD28gAf+uvJ1/y8gekD2wDzBv0DWqEekYLV7L68gpapDXyGosQeZ6n5OO/NZFZmW+Xq1P7P1QHJil6kVpwk927lzX440pwNf3+PRvTbLzlbuVFWJRpiykv+juJyIFjzXLJRF67wB2qgZVm5cRs/GOi4fnvTaou/Ae51mvsO8M3JL5cphApFKD/AO/X+3pHBhLjvaW33cYsp0Z4Rce1nm8wxSmvuJRF+HAfjBlknhDl5ZaJGA96RYA+g9WbAXimFprg4xeiRp7liPExQwtVgackJweoC4MahC1glmjC3eOh9dvrrxLnALOLMOkyCSK/tOF/iq2PJaLViObanPI8mnNkMLbdJyzUBf8hRGdSLcw98qxaf0sSZ+vz4VQS6e9CwX7ikkNNsTeYXjnJ+1yU3mfCW1CXJDr7kBDVG2OTLefCpMY8WBAYDiEaK0RW83cBcFIKfecKYd0qStHRaOsvmyWdFYgBh2vM1UWjro5WXlb+7aJPrUp8ZruapkUXUwhjHA7FwGA4ysE0tsOVVFm0Dw8tvyVVkVNm/9EObG28dWctioZ2+ZEzTWgMJU26Q+wOJ/U4qZyIvpMRUgPZc5nM9J8kcmvNWtb6DV9k/6p8h28kiOxPkxLtPP+3TlBCHjq//mhEnJZ/d8w== shshyukriev@suse.com", "key_options": null, "keyfile": "/home/shyukri/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "shyukri", "validate_certs": true} 2019-11-25T09:08:58.200 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-11-25T09:08:58.201 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} 2019-11-25T09:08:58.429 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxDJtmmcnrbA4AUsmBBZq/M4xNdIbOT+cQQxhUKeCu8F9mht+RznwyaaJbVIlD+UNv9LMhRyzI5oktSalokNltlrtPQMoP52PXH7KFFGnVnexCHAk4nGvEhuZCniO5T+ENm2wFR47lFEQK7lyJJLp5PFiSM7vM1MK6UkMvRmCTRildjaYLyf3EEvBXifbsGmR0tBIXtml6KITCAaT6l0E94055oEGN2G5NhZIvfLcojt5liEkXEHQNdfkUDXM0Go43JZzgsSWDXC/7RYLrAJwz5mq4qCa9YZL/3Spjso37+1oLq7lOQ0TDJUAQXmnW0So/NkzECx4MolVcylhTzmgj sergio@milestones", "key_options": null, "keyfile": "/home/scarvalhojr/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "scarvalhojr", "validate_certs": true} 2019-11-25T09:08:58.564 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-11-25T09:08:58.752 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEOigBGMeElb3uHXX36HxLtPJMG+iG/6D7NDFCVR6AyYsmimBQb6SlSIR6SaFULb6HT/IFVk0Jf/z0HoipAP7Xu+wvP/J6LFoMdguxylh1ov8raqxQJrGLl9n64EI+qWINwEM0MKMUnDQ3tmMKBl78LKQVQTYBx1+9cXsT6DE1yhgmV79XIvOOb+v53agSi/kLJF2i9K04kpi1OejXg95Ub2eLMY69Jm8PnLy1Jq8DhC8pz7nrfzbKtOC7HbytTAO1XTlb90lVhqC/1yTMAUc/rpJwoSEHc2epGeZWdMsJhqhX2EqMwcV53M3qcFSGreGNJ/Ncd4/JHxz7q0IU411t rakesh@localhost.localdomain", "key_options": null, "keyfile": "/home/rakesh/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rakesh", "validate_certs": true} 2019-11-25T09:08:58.888 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-11-25T09:08:59.011 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:08:59 +0000 (0:00:37.575) 0:01:30.392 ******* 2019-11-25T09:08:59.073 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-11-25T09:08:59.075 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.097 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.098 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-11-25T09:08:59.099 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCvRwrsN62CReT0/kOeFEXGA3THhzJ+hwZVpUzo7VMRZeSo6LgNlVGdrQchMHnlwBvuwaVNJlBAkQr+5J5BB3ikrmbcnLHfPutXBBBB6uDfSwwbqjzUY7elTbz9ul6AkegR9K65O44ouD9+WC76D+Tp0maqnP1w0O3h1GVxDHe34LpEwBvZJ+/BSYmBpwRZqeXFRADORPzdtW08PSiRErGHb8j+j5i+br37JfOOh+A/9dHwQw9alBVL2DijwvrPMa7dV5yX00dlMSTL2LLo9IdI4aFPmie8oic0eanFbQDq210ilOR65IGeshKHEKOq0KFFz0nSK9OdO03OkSEN/0d5 ryg@tapir", "key_options": null, "keyfile": "/home/ryg-/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ryg-", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.101 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.111 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.111 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.122 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.133 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.144 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.459 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-11-25T09:08:59.460 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC0zCoXv2Kkxjqv5FOlJZ2PRA9tStdXAVGZJbSLOhAWeqHQL6O2l8MO5GFzHz4FA/g7lbNzLKaCbG1wYGuL9FOt0o36/eHykgENu00SlFunIQn+T/WNaGxrZqnfbdS9TdImFAnVY4vuzrBhFTb6qcqTLhw1WVz+l/iOv5CoY+7NnILWcj8dHNcPzU1ihlFAbvQgQcMtt1130zqc6L8o+SDm2YQF2nuOWxj0LbD6YpRn3AqWsTN5QbqzdeaJiUt6kdywisPdoiSWLuivRKhNLNhnWO+B4DnchwfApy8ovJShHZAsrSoc+a+RQDmOiLXmc5wAvNnREHbqod8/0bQ9vSXaVQCt1gnnK8Bf7GQElg3Z3aep4D4w6rBB+LBsHFbTfrzWAHkXxJeGiQTlJGz0IYHi1weWrzY6x5rEA0gYDu0FVLOLrsYDtuTQpjcTwpmaHCsxz6wU3wCNmTsV/AZhniFHJhHXacCmDp1JfsI2kP9FjcJJE0SygnZ/SzKe0DHamnyT1OgdQaG/KX2eRv/vXUPg2ukKiA1fyFnFeEsWK6rUO0YgCswqjrM4vU0leZ1b8vK9aM4IDJM3TJqdMLfOa0rPntekNOqXUn213CP+Yiz9l1TH4zX3P/Kv6Vv1gTeGcX0lMaYhowng+97j3LLqGcSUPMklrKen/xrsACFxN9fxaQ== prsrivas@redhat.com\nssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCks+XmOepYEdSBP6U0T+r+fZowSS7UHY5W8AUtAkvXiKlTG518luG2wtSALIsB5+jyjC44ITwV9X/vvsDaXQnp1hFanwWeuU1jDdy+MvywvxI9q4naWlDpxrfq4i/AioSvdoIo02uUFYPKTCHrm7wBjTwpWbCD0gbC9/OGAHM88Md3F0qbgFEr3BERvNGe0coCjWRMS8+GTZXS22WW1/NxNzpdCUjO/9j4QydpW7UF1kqVhnPVZozmh4wl0rk8tgb9Zx3vCUBG1Q5eO/q+HsW54eXLa9FcuQcoJnNdV2hMXRHHZHTgaPKZ/ildukzZHo0i/Dq1mi5N3s5A7aDYxOWWlRXazwL8zu/UQVrxVRni8vDTFk5682lskF1WifzADaqicUWzasdK0PC4G7JmHPi+XHskeOt5i4ywEnFRXnYtMOIu0CNO82Rg4CdUAD58YXCGoQ4r9TG4LlwKg75ouNHB/ToJidSg9NBwclP+MICcTpWItxVXpBj/YkeLcntHtVTFNHORxIwIwWng94wVEDCfyXelyAR6VfwcPlWWrbzhbEO9DH40GuluQi38ySWi/HpnrVriJMLzEQA9OPrLsME1LDgPGkF2GloPL8nnnLWB9L+YEBNNAVoSo0PHBsjIibdZ/SZm7zUgYGp4fuG+m1+SDOjcfD6GSXOTQwQou+knEQ== prsrivas@redhat.com", "key_options": null, "keyfile": "/home/prsrivas/.ssh/authorized_key 2019-11-25T09:08:59.463 INFO:teuthology.task.ansible.out:s", "manage_dir": true, "path": null, "state": "present", "user": "prsrivas", "validate_certs": true} TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-11-25T09:08:59.464 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-11-25T09:08:59.466 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:08:59.790 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** 2019-11-25T09:08:59.792 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDlnlBzRdWRqns/9VrYuF9tLILUjeGd9z8Tf/dPZmxAt3nnZqpTvI2xXemQpyt/2lsD0U24EuTc1Y/dBsgqvXtZOw51TZE8RK26hvCCaKGfcKONjvR7goHAmkYPHK9+F6m7fgYkRzA0+ZW4HcWNh/xxhhL2G1EGfz5bzIoGQJHSdzc6VeVIsY7HiYEMTxvxdsgV72Kjq97BdmHm56RBzoH24oN0UzNZS8Syx5SuWj9dA3IyFBPwjLlozSk6U2Ofd7qaXuQKl807eD9g6RhRldzHPS+i0EK3EM6Jvho5Y08k1KqV4gIEwMdzDbD3SH0xq+nUPFKcDTUFNp+tkg10GxqN zac.dover@gmail.com", "key_options": null, "keyfile": "/home/zdover/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "zdover", "validate_certs": true} 2019-11-25T09:08:59.813 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-11-25T09:08:59.815 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-11-25T09:09:00.083 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "key_options": null, "keyfile": "/home/sbillah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sbillah", "validate_certs": true} 2019-11-25T09:09:00.105 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys using the keys repo] ********************** changed: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": true, "comment": null, "exclusive": false, "follow": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQC4zwWf+kMEDgaSOrX7AlD+UCOgf4/fRiorQVJqC8GfMNynfiFVZaThngwIrwtQROfXHbzspi+AcRvDLewY25mXyuxVgOfSbuSIEqiUNvzcVq9IItI1J58xeDBpIktAsoHHS6JENV56fAT6krzRAQLnSa1k2b0pyKuPBzFSeNgiaQMhjrCLAaCi9yLsdKpnWgXhyfrdqvN2r/Ctts/zlTPqtRYdvdIL/p/l+vg7Q8S3OhuAGtimaEq3RE70xadaPbGhsLISMTrbfYXbxw84KKZpNYwO4zmQ2ztcvC9D7A/8ov3+A5Jq42G+pq7ORuix2NoTWtADMiwvNe1HyzS/7cX/BuXLFssRY1bzRvp+x2ExZ7b+bFx+TE/RduRh0eyDNLg7SyL83aeTr2Usa+mPoCAMKed9XtYbRnUu7ox4wmfeAgx6jM9uG5uQfauJ1XpDddrVBG8KYBvSd0BM2cyRdcAXqwxg9AVPZcdE7sbBk44CGVhw+ziZ3e3cLBTO4y44KRQre+1Bhe/Ze8OTQWLFugl/XagFlJE/L6fGUIPuFojmQvlw5dBZwtUURnxsu8yHm3iIsj4y8nR1zHMj+zuMk6rF3ZW0epPAECAdq4gG8X2wv3HTq0vLBkKpi1KMAmntEtcVwF0fzMt5XYqRd4jceFgonAAipJNDknHgi6yaUzBYmQ== rraja@redhat.com", "key_options": null, "keyfile": "/home/rraja/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "rraja", "validate_certs": true} 2019-11-25T09:09:00.343 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:00 +0000 (0:00:01.331) 0:01:31.724 ******* 2019-11-25T09:09:00.377 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-11-25T09:09:00.378 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-11-25T09:09:00.389 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9', u'name': u'andrewschoen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "andrewschoen", "ovpn": "andrewschoen@home hYDlX2i7X8t0JHpK2smvFA 2a6e753fb3c401ad75e1273ae5e18e4abc0404149dbdc3675bb317a63a4469e9"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.390 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n', u'name': u'zack'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zack", "ovpn": "zack@ceph0 1CeZnO8st4CuhLpH/DJNVw 7230fda728e34998da4401284fc3e5c8939bfdd52482f8cb675d8acf6fcd12b4\nzack@mba.local mzf7/v4tv3uVExt16N/w/Q 13dcf7082b974315dfab1c76885dfc094d7ccaf01ae2e7898a20005602be2caa\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.401 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd', u'name': u'kdreyer'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kdreyer", "ovpn": "kdreyer@redhat.com yUBHWOGnPts8xuDSon5ODg 38feb4fec9f1ba3de74ed074d745ed46a8e5b3ac1d44e50005a04cf07f0462cd"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.412 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47', u'name': u'adeza'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adeza", "ovpn": "alfredo@papaya.local V5VkRT2Du88WwQrYeK196w 4dff4d9bdc4d9ab60532814c338abe4534cb45a9ca729bdbc2411ad513516b47"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.413 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n', u'name': u'dmick'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dmick", "ovpn": "dmick@angus z8fyazJ0pc56I/XTlHAOrQ db682b1b89c790bf26a2e12dbea03f4495f8ea5f248e29b2f5c5dc072acb3c93\ndmick@home o9fblAdxhBToEJxbutv2mw 1704b16322c3984ed565b48cce52877eadade26b4093443b349b70f846f0eac6\ndmick@dell r6meTomZVVH/WVOmVf2aSQ 8ac1a9d0452166af59894c6b57ed07a12b42042aa6d2618df2bec9c962684072\ndmick@dmick-MacBookAir EFN6/5V1I5YNm72XjlApUg f2335a3fa49a0296233ab5faba28e7bbf309a89c193169dbe3c411e4219827a4\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.424 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n', u'name': u'sage'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sage", "ovpn": "sage@autriche 2Uhfc/1jXje4W7G4s0sJEQ 09631754d059fab96e16be441b3b8f7e63b37a260677897a2669cf7a2501d2d3\nsage@gnit YuLp2iihT6cCqY+GmW9IVQ dc79b19c45b9be1b4e69cd7debeb5b87aaa8cd0050353f798913da22c233875a\nsage@maetl wpsQIdtAb+z+ietqr+90Zw 42c518015b7cd440e7c8afd65fcca5769e31e9ef8f8a67d109a3de38187087e6\nsage@lowride tak6ngfj0ohW6upHRUMq7g 450919c617f452e874ef2a8cfa5ca18f47d249450ef0b0374849e721b0c53a70\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.435 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n', u'name': u'dgalloway'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dgalloway", "ovpn": "dgallowa@laptop POebac0SJIVkSCyi0LE4yg 52fc0cb34f3332a5f3cff117a20f00e8ea89a051736cb543a689a96dd9528ae5\ndgalloway@p50 7PHdbcNE/YjcHOm5NbNfpQ 7075211c35997b01797c2f15083aedf13888fc4c54080130342dccc76469fb40\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.436 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045', u'name': u'akraitma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akraitma", "ovpn": "akraitma@aklap 2zvxeeQ+bQ4UhKBE9Kz2DQ ff4a9dbab841ee62e7328d8cd0d285c68c22a1de6e23ea24f470b3dcb1a27045"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.447 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e', u'name': u'trociny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "trociny", "ovpn": "mgolub@mirantis.com +WMPxNCgiMwhNJAJHNCV2g 4cd6749ac740ea6dfb50d4f602920ac75e979286439bd510540a8a551728104e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.680 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-11-25T09:09:00.687 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.698 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.710 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.721 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.733 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.744 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.755 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.755 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.767 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.779 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.781 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.791 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.802 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.803 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007', u'name': u'smithfarm', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "name": "smithfarm", "ovpn": "smithfarm@wilbur vEk7281ao7DkpXE2Cn3rpw 20058c04c4991caf38b9517faedb75c8562e8d8669808b6be9f70da6a5b64007"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQC6jqQtqO7pnW39mpSQQC4L2FuxPOx/5j34lp2PrwuTWJoDwtUVCZhpNwSs5UAj3wdXGXCzWYWxu9d9Gh0UEBkiC+2ZHtLvWrwS1TDVRy5g5jYVm9vX3kL5dhoC2vYPyQoQkxeMX3BYlhLwkSjCAnjaceJJZyOT7KsqbpanvJxvudMzeyCosntnMuxyPzjH9CkVKqycddZnscdHMkDIZ3TjNhnL8GKK/QCPB60tkCO6HWN/g4HxidRjrp2VjQgIa5MWN9LBR6mGspZI6+rR1ygLhbl1scr1cCCumh65r+zmCC/ZY33YCfRXdmQZo+ShsBa+KbcvakEx0Bqgx7DsMoiR ncutler@pantograf", "key_options": null, "keyfile": "/home/smithfarm/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "smithfarm", "validate_certs": true} 2019-11-25T09:09:00.805 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.815 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.816 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n', u'name': u'mbenjamin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mbenjamin", "ovpn": "mbenjamin@ferret N7Ln6KHUAYT/iiP4Swkb0A 244b90574d7e294f6f80949ef5e640bab103c2c34e1902a69cdcf592607d950c\nmbenjamin@duel PRRFsQiomHSJX8sVgRHblA f3009adb7502a6c5554e1157d36c63d84f40fe5074a01d7d0280e5c48da374dc\nmbenjamin@rael 8ixfNNx3j2m2+nGhXkHsnQ 62fda7ec724f0fdb3f05add4c3739e07a5c7eeb2380d5f1f7eeeb13db35e16a8\nmbenjamin@shatner hJ1oEl5wSZaKPJC1Ugss8g 9c9161b53da4d842308f80e5b490a0c1539076f129bd0e34ee2253272dd55c21\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.827 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.827 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.838 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.850 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.850 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.861 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.873 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.873 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.885 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.896 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.907 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.918 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.919 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.930 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.957 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.962 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.973 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:00.995 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.017 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.028 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.040 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.062 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.062 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.074 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.085 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.096 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.097 INFO:teuthology.task.ansible.out: 2019-11-25T09:09:01.116 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.129 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.141 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.153 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.164 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01', u'name': u'aemerson', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "name": "aemerson", "ovpn": "aemerson@seasalt enqtMOCQ0a24hizRK46SZg 42f36c9c14c1eb7c468e41cf4c5649e30037f0aaf5eefbd6c07be9637224ca01"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCxUDk9jEQOSrRwe0WMGBPw28fzgOhvqMgcG90SvuUjR20Rx3KUfy9JMxPJPesjBQz0xgH5adOVg2JQeLrt3uSJdIFNpqox8BBrS1u/bWT2dorQjLnFEhrtA8Ae/h2kwLQ8w3yYNWB/PxDSuMRnyH4G8EWtVydzQcV/QnNZ9Y6eBcHaI5C2yVnFl7Fi4zBYc2eeL8o8gWqmM6DobBCvVrxD0pCNAsLhOW6IA140BNGT9F/hQZYje9RQRWbFKh/iKiUhqFYCzLcxFfYXkw2HZMJA2p/bLmuc8ZbgYaIiU6b90kpfDB37Xw0S6toIj9E8h+E3nkNnwraCQcbralhz/bdz aemerson@seasalt", "key_options": null, "keyfile": "/home/aemerson/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "aemerson", "validate_certs": true} 2019-11-25T09:09:01.165 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.176 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.177 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.187 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.198 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.199 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.210 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.221 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.222 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.233 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.244 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.255 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.266 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.278 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.289 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.300 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.301 INFO:teuthology.task.ansible.out: 2019-11-25T09:09:01.312 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.313 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.324 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.336 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.348 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.359 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.370 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.371 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.382 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.393 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.408 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.416 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.428 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.439 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": 2019-11-25T09:09:01.440 INFO:teuthology.task.ansible.out:"Conditional result was False"} 2019-11-25T09:09:01.441 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.452 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.463 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.475 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.476 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.488 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9', u'name': u'sbillah', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "name": "sbillah", "ovpn": "sbillah@syed-machine qVcw+LuFQQxYW7QpzZ3aLA d028c4635289a781f3ebe26a545e084572613b03cc9cde7770018ad0259e4dc9"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDEgeHHS5jx9k7QdOEGqZjaxEAPJ6vv/WZXJNpifBpm9Ba1FRA1U3qHV8oX/bBZ08HIBGg8hQOeZ5I7d5HyzR0971W6KVgDF+s6mRN7d+heNi3XmikbJrozLoEiVQNHIsXjUoc655c2y9NR9Lf5FBweSSrbE34jCUqTA3XmZOdbkjY+ngOcDIfNixRG0yZ57p6UqYW0I+Mg68CB7N+Lv4gFvH/968aML7ConABPGs+vnLdNSQbjuibnaoZwzeSgPoaBJEqBCgNkwO8TyaC04okMj2X7/FGxgZNhwF0V5SVpBllWlGqdAigEF0dher88PbzSIFSm/x8PeACSZWkU0QWV Masum@MASUM-PC", "key_options": null, "keyfile": "/home/sbillah/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "sbillah", "validate_certs": true} 2019-11-25T09:09:01.490 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.491 INFO:teuthology.task.ansible.out: skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.502 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.513 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.524 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.525 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.537 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.548 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.559 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.571 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.702 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.704 INFO:teuthology.task.ansible.out: 2019-11-25T09:09:01.795 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:01 +0000 (0:00:01.450) 0:01:33.175 ******* 2019-11-25T09:09:01.822 INFO:teuthology.task.ansible.out: TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:01.865 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:01 +0000 (0:00:00.071) 0:01:33.246 ******* 2019-11-25T09:09:01.876 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78', u'name': u'amaredia', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "name": "amaredia", "ovpn": "ali@freerunner yQjRpRVG7D5KN2HAUjI30g 9d677a34ae98477e6cc8ba1d975d81dcae43a102013b265c63f3ea91e7dacd78"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCc1ZbHNXuJn7gxyt7QJ579gEM13MuFdTzsrnedYuRIW2Zlm4rCFr8Oj5SGs5DWBIJnd3W4O2v7PjuVQMxU/zbwnj7mdwBmLFe1cSzOJv2eP1R2uaU5z5C7KNmyPLU++pGKClzb6v5wcEQUq2K35xnuXUU9B935dK+Fm7bK7+HAxj+1vpVeycbPFyPhf6mwbx8dZv4uvZGV2+CGBuyIB/5U2AMJZy9LWim3AR35bip4ftXvSKlAON+RHhnS0toG/6uwp0XlFuGn5H8snaca7L6hGtB4xg1PqA5aMf33Jiv2NVLQo8emHU9J/HeNVS7ksoSZ6InynpLZ6b9uXa9OM9XL ali@parkour", "key_options": null, "keyfile": "/home/amaredia/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "amaredia", "validate_certs": true} 2019-11-25T09:09:02.193 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5', u'name': u'tserlin', u'key': u'ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet'}) => {"ansible_loop_var": "item", "changed": false, "comment": null, "exclusive": false, "follow": false, "item": {"key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "name": "tserlin", "ovpn": "tserlin@annarbor DlKe+OWBPcFAQtWMUAHnwg 6b268bd737ffa5dd38865575ccd444b92cb912c70f5b82dac41f9c50505df4a5"}, "key": "ssh-rsa AAAAB3NzaC1yc2EAAAABIwAAAQEA2ok6CBUpOVGv2RFws44GtEP5SVxOi4Vie0WSZYLpD55rTfmOtsItN1d1EciVNTUyWuyzMeQGWC4JAd3/2l3gR/5ZwSvd7b/7TCNYxjAMEubRand0GxEoiKhpkJMMmJqcT0KefP8pr31MASWPuERj1+0/IbjJExsvrJaUjqeIfZ+DWR8dC2VYdcH3hsp6AE3mqKX/9693sxe8ROt6qY4WkpZcO4M90unOVa2CnJsYqKaaIC4z3fmKuHZpJZjiJMrg8rtuN4r7bnKWPEVGcahj+i74JWwKR5+2gntLpxw2chIBmf4qFu6HDplddig4V3I/2NLB8soBpgc+m8O7YyYl0w== thomas@easystreet", "key_options": null, "keyfile": "/home/tserlin/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "tserlin", "validate_certs": true} 2019-11-25T09:09:02.205 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28', u'name': u'dis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dis", "ovpn": "dis@zambezi wXYUFvWPBlkCFz+mC4RD6A 11c464dfb2a27986e029f1915732a4f237baba4eade02bb045c8f0d13dfada28"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.216 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n', u'name': u'gregf'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gregf", "ovpn": "gregf@kai YhNrPfedhZjbvGjOfmotOA 440cf8595a87cd307790bbf79c3668455a0405945e2b271d873325de222cd72f\ngregf@pudgy VZrk8nWzg7pYLOrZru8dBg c1d1a0e469a134ccf5f5a5525631a6b83efa6970beec3b23809eb0daa3dca47f\ngfarnum@WorkMini2 +bAqcTdU7Ok9bGMcB3A84w 3cff1326561a23cf81dd6495373cb83ed149cee026c6374d72c19b483f4f1f07\ngfarnum@Macbook bxVtolCC9SY3QNlpx3cE1w aff8d28bfb4d693253511d29e8d399196e964fc096594ec705748a5469d44654\ngregf@fedoragreg Jdn8I/sFGcr5Aa/dici6lw 50f88afc35c05ef8454742226f7baf2cd20cb1e2d4d0c9f4a393013877736bfa\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.217 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n', u'name': u'joshd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joshd", "ovpn": "jdurgin@glarthnir ElGaAgbs5VZujuzsQMfCmA 2a156febba037d02d1099bc11d1e697d34300b2c420f2df664b5b0de1248f983\njdurgin@new-angeles jqa015PRJcHSp5WHcwJjUg 42113e1156382fde866d691f30584f6b30c3dfc21317ae89b4267efb177d982c\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.229 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n', u'name': u'davidz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "davidz", "ovpn": "dzafman@ubuntu-laptop-vm NY9y9tqLY1beEcXDwMavsQ c869d42fae1890574577a8014d97d1247f1a13cb6337037d2714f1d236fc65d2\ndzafman@ubuntu16 2a0rAy5QmNFSEcATNz2h9A b7c11fbb0911fc4ac0216a1a8eac8359a9e8f43d69126db6b45cbeabd358c2b4\ndzafman@ubuntu-1804 PN1pkeGHGloB0K+IZrfB0g f1c01b447b9ec3fc048c32f606a33fb488ff621e11aa305ac979501030202658\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.241 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-11-25T09:09:02.241 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-11-25T09:09:02.243 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc', u'name': u'gmeno'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gmeno", "ovpn": "gmeno@gmeno-virtual-machine FKFu8B2pMqotpmEVAO1few 8229574e499eaf767a408909f5afdf2e2a0bb8f3e61b18d63a651f7102c68dbc"}, "skip_reason": "Conditional result was False"} skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1', u'name': u'ivancich'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ivancich", "ovpn": "ivancich@ann.arbor Kt2vxZ3Ge609mHfjx0W4Cw aaa55a9e2b5276b62a21cd3c401b365c5c2693e39efccb2f9edefafefa1dc8b1"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.255 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n', u'name': u'wusui'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "wusui", "ovpn": "wusui@ubuntu /nElLTGqxiOr/hD6eziF5A 2e244e2b62fa42dadf3a3a1dbcc29475410cee6550b0c6b3603c135853937551\nwusui@thinkpad tu2DxDcllIdwb5ewldgT0g 1590a7d9f1377b0094e9ba4277e7bcbe6374791f0b3d3df93026345c058c93f5\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.269 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304', u'name': u'zyan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zyan", "ovpn": "zyan@redhat OP16IkReCatMfA4Mf3pkdQ b0262be71ef008e2f7d585e34431dc2757c1e22ac1aa844863be533bf873d304"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.270 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n', u'name': u'yuriw'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuriw", "ovpn": "yuriweinstein@Yuris-MacBook-Pro.local wKA8mCcsdsk/CE+1d9GiPw caaf5bf5eb31ef269e3d0bc34d45dd761c0bb0907172eea6e754434de4603db7\nyuriw@home 02TZyR3JHJMxEQob80ICNA 85b4aa0f69f6d469dae0bb3dca4baaf222e164927ed7eed2082caae8f4717e48\nyuriweinstein@xenon1 C9eVdLb/i18lMcMG20rGPw eaddd0e831a77de3f35cb19e307bd6f9aeb0cc03eff1ae58490d7db33ced4311\nyuriw@yuriw-RH 5ivdxgFO4eIkbXVhl8xkvw 59212d29b8b42d9fe457c1b2c43d774e1d25807be10dcc1252d4aec63b97a467\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.281 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n', u'name': u'tamil'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tamil", "ovpn": "tmuthamizhan@mac /1CBLC6rxqzJzPspffZV0g 4d1dfff2e097a7fc2a83ea73eccad2f0e453a6338e18c87b4d89bf370733e29c\ntamil@tamil-VirtualBox M22QdmhkSPj9aEcTiuIVfg 8e76e06b14de99318441c75a96e635a92f5bddc54a40b45276191f6829c6b239\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.282 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n', u'name': u'jowilkin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jowilkin", "ovpn": "jowilkin@jowilkin 2r8xOv/eCTcHr+HSsMkPYg 0ac416d2dc139628144dfa0822af8cc0a455f5f5f3e4d1b9713c14115c062218\njohn@osd-host 7zjDTxAYhCmTX+Az4SJaoA 7d924233fdef168e2c5c01258aa349de108629ef2ff90d17c0b96acf22dac7c2\njohn@admin-host 7cpk7iJ1Hg2vk4bPDovKmA 05765178f27af6dc4e43e47f52d773aac3bc1b3f1dd998bdbf479b951bfd2efb\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.293 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4', u'name': u'bhubbard'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bhubbard", "ovpn": "brad@dhcp-1-165.bne.redhat.com 4oShQI9+vYtX5gA47np/Sw 3fc7df5afa772752d8eee15c01d550cc1dcc88b6e940abc9f9f8f26102d239d4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.304 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n', u'name': u'yehudasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yehudasa", "ovpn": "yehudasa@yehuda-2 NGXHeO0lAvRFfYLalffzEw 0d3489b09f48ad5fa9d1ec5944d9c8020daf852488f389f3d44fe63d3f651f34\nyehuda@yehuda-940X3G shisK5LjI6fr3ZBJy/xX8A 49522899bd26130086ce668079f0062d987d85dfa5767dd5c34e5953db97997a\nyehudasa@yehudasa-desktop OT1MhoO0WihhvkKztqW0Uw 12a4d6b54390b9df7f5af3bd6b533f3c1fee0c7b9fbb79f0a87bcb28b182c7d4\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.316 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4', u'name': u'dang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dang", "ovpn": "dang@sidious w0CNW2g9K1WiRenVGYWNUA 4f59d761bfab3659115da2b3b80a486266f77b09d8527983217d15648b4f92b4"}, "skip_reason": "Conditional result was False"} skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb', u'name': u'branto'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "branto", "ovpn": "branto@work ganye+HpG3dkMEik6WtTng 018f3f9b9d49dcefa701ea304a8e58f002c46f0650edae220a0a7ab1bce36aeb"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.327 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6', u'name': u'xiaoxichen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xiaoxichen", "ovpn": "xiaoxichen@ebay RvJJ7BhIehpoPtggrwnskQ 862ecfe7e15dfab86d61df86856bfe06cbb99f240f6f03851f7f9e1a255327d6"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.338 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e', u'name': u'ffilz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ffilz", "ovpn": "ffilz@redhat.com 6YdvqxkKfmDWGD2s0wA7Ww 4ce64d08686e34e559ccec2498df433b155b70c9ebccaec616b6b34f0f0c246e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.350 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n', u'name': u'joao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "joao", "ovpn": "joao@magrathea eSS2/gvK7ALE6L9bITfuMA c3caaeecee3f43e39b7a81fad50e0d874359c70a9c41b77c661511c71f733909\njoao@timesink 9S3oER36HheVupjRpnLz6A 9dbc964184244e9da269942dc73ec9ebba6594bcccfdc0eb09562b58b4542162\njoao@tagide uUDVmZ1S5lp/Hx8eKTfDCw ee37985d0a51360f08266e0de208ea85616777a325d498d7c895a80d4723f844\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.362 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n', u'name': u'nhm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nhm", "ovpn": "nh@latte JCH9icAtDPj951rgbPKJyw 9a3d30ec1ec9467ccdc8bdbbfacffd7396fb38e046199ae37b2b7b69dbf37480\nnhm@espresso +YYZPT29wYzY5ooaRzabCQ 1ee041dd58b9ec6eb678c47632ece7cf6c24e23bcbac28a77a82af05ba6cc148\nnhm@mocha HgOGOfkBEzJihFsKmPRfKQ 2e17f3ba0b90df7a36f19a7c8f64d2aa8f966b2794c94caa110d313e927a1c1b\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.364 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n', u'name': u'jj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jj", "ovpn": "jj@aurora PFzIfG7NjrCYxJvxd//soA 395884ddf2caa0043f97e4f98530cc3ad646d6f269802a2b79ca1ea7278ee006\njj@metatron iQogIw/KQtewT7oj9Mkivw 0881e5ceb5897e8a370bacee69ad58eb5098090ea4b0d53972214ea7c751e35a\njj@laptop O1e31whZbQ0S7MUtglCRLg 96e39257989ce36e240b5d368e0308d38009d3d923ec398dc9cc6eba371acaa4\njj@aurora2 EtAvlrozxiL3PLYp6mvATg 1018928736c33ed06246f208cd02aa10c0a6efa5b4e34e32408d7a6c72c32e11\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.376 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313', u'name': u'nwatkins'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nwatkins", "ovpn": "nwatkins@daq unHfXkUpAHOnzeptznuLLA 33b2003c30d0cc7a6b194e76be92c7b5d270c2d2a4b4a8b6e673f0f0dc1db313"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.387 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22', u'name': u'mkidd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkidd", "ovpn": "linuxkidd@zenbook oYp1WWV0JwpikHCWCV52Lg 9aca455b601bf3a365d31068154150ac63dd76f32cef29a55f9685dd1a88aa22"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.399 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af', u'name': u'jlopez'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlopez", "ovpn": "JCLO@oh-mbp-jcl vZhmBh/1LjLFEu+atRec6w 1f13f591373b4dc798a9b701fabf1eb99bf4aa58f87b6420d6c916716f0965af"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.400 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb', u'name': u'haomaiwang'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "haomaiwang", "ovpn": "yuyuyu101@desktop XAUS/1Geh1T2WY//5mRahw fda03bdaf79c2f39ac3ba6cd9c3a1cb2e66b842a921169f20a00481a4cd3d9cb"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.412 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608', u'name': u'jdillaman'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jdillaman", "ovpn": "jdillaman@ceph1 kQ96pIpKTocwIj9H1Udb1g 6c087451af94ac18c144712bcc9329799d86f6d90376839dcd6fa41cd73e3608"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.424 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3', u'name': u'kchai'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kchai", "ovpn": "kefu@gen8 HVoNrB5C8+bYxuqIfByeEQ 4dddde1890af2d6df367d3d832cc3b9b660160a1db69f0135e0d09364b2cb9b3"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.425 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@vumrao VaMHdnIGTl6y9LIkurxfjQ 71de53c4a0f212b8f919437d7d433d24a33d7a33bc6fe5df5d047e24499994b2"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.436 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n', u'name': u'dfuller'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "dfuller", "ovpn": "dfuller@laptop 6U0lNVidRy7Ye/5q6cGN1A aea3d019f68a95094c99385aff099818224455a829615cfc774587e4519398a7\ndfuller@rex001 6Z8bfQDgPXVSGMeeQHoItg 3abd41920b72683fbba7f25be88ff992fcd753119c4d2086c12daaf20798e684\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.448 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n', u'name': u'owasserm'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "owasserm", "ovpn": "owasserm@owasserm.redhat.com hsdVTbVub6eRnhlO9B02rQ 7c9baf41670ff9ab612f75d4be42d0aaf0d7ecaa3c8928032b61f1be91725890\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.460 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9', u'name': u'abhishekvrshny'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhishekvrshny", "ovpn": "abhishekvrshny@flipkart QTqbWHaqvXwB+yBy6CVO7A 25d026c49dc49b3a1f445d2dc0099d5ed916645b0adb8d0306269ace7a2096e9"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.460 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n', u'name': u'vasu'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vasu", "ovpn": "vasu@ceph1 +1D1qNUAk8h7OF9LF9qkrQ 963aa256fb99bc4b77e7085f57cf910a04c7f143603f81252331411eb8e37ec3\nvakulkar@vakulkar.sjc.csb O8ac1k0Dh3xkIFX8NFyIZw 471538eeb22384b58921e4f11af272c00c0a953dc7fe8d95ba057e65d141fbd2\nvasu@vasuSrv waJqYAARY/LnfuP1x/KQzQ 68915d3a1eb3dd00a562c149791cec5f43a96f5fd0b851ec855ec3f5dab496b4\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.471 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37', u'name': u'smohan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smohan", "ovpn": "shmohan@laptop 7wwHLZZNa4ShUV1imXDDjw 0aca19a8ff6dbeee2821dd75a329a05b8052170204b2b242ced9b1a68ca8df37"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.483 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114', u'name': u'abhi'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "abhi", "ovpn": "abhi@trusty SKarQTpBigBobP9sLjdLiw 868a74ed21b46f7f64255897d824f4e3eb21f8dde844bbdaa386681c942d8114"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.494 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246', u'name': u'shehbazj'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shehbazj", "ovpn": "shehbazj@smrhost oEliXj/jPmfXv9AUAFKB5A 8bb3a682b4ff15de655c0fe610d350c5805d0a970471e4810b648f47e2811246"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.505 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611', u'name': u'cbodley'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "cbodley", "ovpn": "cbodley@redhat.com W4gFS4oU8PtUOrdHuBYwXQ af7dafd992687f5d85a79866838a78e5070a4934fb0a935e8094adb31ec28611"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.506 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3', u'name': u'fche'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "fche", "ovpn": "fche@redhat.com LiziWkWg2uWoEHb3Ln92dQ 9c5497793758b069adbba9284dd55944276ba4dac0bb95d9357c81b58174a3c3"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.517 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4', u'name': u'onyb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "onyb", "ovpn": "ani@orchid AgbLf9Dufuji5r+9WxM69g b73b7eacb9b628387a17cb1e0a84ff19c29d45dca8f0768e407aa599bc6996c4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.528 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195', u'name': u'mwatts'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mwatts", "ovpn": "mdw@degu IPiZqcFT2BLuf2h3+tw58g 7af390a631ec11bddd7d1ac506d29af65e1e01e19f7dc931b4f459030cb7a195"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.539 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3', u'name': u'oprypin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oprypin", "ovpn": "oprypin@bx-pc-arch 5WYD7PWKwdSQjfGo95ehWw f3c9f170d74c6c443cf0f82f0d87045e1d3a0dbcd01474f6e667ea20a00978b3"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.539 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "pritha@dhcp35-190.lab.eng.blr.redhat.com GCk3PIYzUNZ6/xrsKoq8VQ acbfc4279970b44c3d008990e0cf1bb5eb280299218441a0f25fda988bc555f6"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.550 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea', u'name': u'pdonnell'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pdonnell", "ovpn": "pdonnell@redhat.com Q9j56CPPXipXScmKi57PlQ fb616603b6d27cf65bfa1da83fc0ca39399861ad1c02bfed37ce9be17cdfa8ea"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.562 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868', u'name': u'jlayton'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jlayton", "ovpn": "jlayton@redhat aNfzMdXOhhmWRb25hwXJIg f51fee42c5268f7b8e00d57092dc522b0a07b31154ea52cf542da9cac5885868"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.562 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f', u'name': u'rzarzynski'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rzarzynski", "ovpn": "rzarzyns@redhat.com h4UkibgFG40ygfpfKTMBrg e20ca28c60144dbabc97953cd4c273c1b92cd45ebcddd0f0299679d7a5c87d7f"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.573 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** 2019-11-25T09:09:02.575 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-11-25T09:09:02.586 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** 2019-11-25T09:09:02.587 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e', u'name': u'rdias'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rdias", "ovpn": "rdias@rdias-suse-laptop 0bh54sdB69mh95n5rWME5g 452e3338e48d04d4e816f4f1cb54d637746a7acc1ffe5e8ed4c1506c8e07a72e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.597 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051', u'name': u'asheplyakov'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "asheplyakov", "ovpn": "asheplyakov@asheplyakov.srt.mirantis.net wFW0ZgT4cNhKRAGXiUtevQ 1b11f0702b2db42a42aae6579737ece2caad3b80a8186b971686575cb76b3051"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.608 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953', u'name': u'vshankar'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vshankar", "ovpn": "vshankar@h3ckers-pride r39frQXXj1GUJZwq1GS7fw 1170ef4c918c5ff15334d10f666441b0dfe0bb869a5e15218fdfad2e8cc4e953"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.631 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58', u'name': u'akupczyk'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "akupczyk", "ovpn": "adam@TP50 C0YuBT9bYaNhdDmjbF56xg 5d298b33b9dbaef364b037561aa5c5de374405bb8afead5280db5b212506ea58"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.632 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e', u'name': u'nojha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "nojha", "ovpn": "nojha@localhost YQTw/h6ZMgdJn7EPBmbEnw 253574eae62759c4c5d3bc4bf949c59948a0488e4dfe4c91ee754a3b5494847e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.633 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43', u'name': u'ifed01'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ifed01", "ovpn": "ifed01@LocalSUSE1 6g6hX1bzTGBTCnDevAn0+w f18c9354f6de3f371c3b51521b62375e474802ac21adb3d71e09d8d5bf9d0c43"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.645 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4', u'name': u'myoungwon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "myoungwon", "ovpn": "myoungwon@omw-dev 78twXwYRU+MeH+yZ9Rw9Zg 2dd66fa122e5cf3b8bfa835cefac7c6e4e66d70643a3819813104c2057e597e4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.657 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0', u'name': u'jwilliamson'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jwilliamson", "ovpn": "jwilliamson@glaurung +d028NM7xCxkVdxxO+b1Lw fece65125073fdc287af724ee4724ad84d2864e758d50dcb23c07b05c3595fe0"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.668 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2', u'name': u'gabrioux'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gabrioux", "ovpn": "gabrioux@elisheba 80kx1htp39RsFrlGONcp+A a95579ef6f90694cd6fd390302adf8532237a8ea65bd5544d9b561654d712ba2"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.670 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f', u'name': u'leseb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "leseb", "ovpn": "leseb@mbp cCx1v5/FfaQ/IQHujqtG9Q 6121d11f9abfa6b1b36330eafaa2196249a9c92f989be25c9fac1558292c920f"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.680 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca', u'name': u'hchen'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "hchen", "ovpn": "hchen@host12 hUr3k0rugStZMjvIxIvCOg 9d57e14d49901f18b24ee4076ae7e6a2f9eb6fd9fbce786660c448486c966fca"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.691 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n', u'name': u'jcollin'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "jcollin", "ovpn": "jcollin@earth +H4Hk4WcNuqdQj7ch/Nulw 8426545e6457c9e1e8adca2af5ddf836fbcfb433cdc5359fd135afdf4e0f7d2a\njcollin@stratocaster jbjV3FsrsTJwyKUA3Y8VVQ 0439745f795fef1399636bd550040d45445d1607b471284c5c9b9dbccc86a987\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.702 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc', u'name': u'xxg'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "xxg", "ovpn": "xxg@zte Y2d/Ov201XMivDNwo4nUoQ 5e5da8d579793601699af628300430c1e5dd469c8bcff7c3ee11d23ec004bdcc"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.713 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53', u'name': u'pcuzner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pcuzner", "ovpn": "pcuzner@rh460p oK28wU5DSabvEL4VjDRhEg a449ed81d7e2970f418263fb3ce10dd711d03925a0990ddf298f826aae1caa53"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.714 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77', u'name': u'liupan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "liupan", "ovpn": "liupan@ceph-dev-2 9+Is4mIZgNkYyJLwHvSNOA 5a8fafc187d52041daf4365125692d4619fc557b75560913130c0596f83bbb77"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.728 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2', u'name': u'mkogan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mkogan", "ovpn": "mkogan@mkP50 f/LENRJbsyepmvZA23F7Fg b908f1c0237a7ee56b73dc42f2df79b49ca83d6f4573f5229e7cfe6b4ad7b6a2"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.741 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6', u'name': u'amarangone'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "amarangone", "ovpn": "amarangone@macair.local 9Fslt44BqONCYNhf+uhcnQ 12d46ec6815378a12abc5df00e65235ccbc06ffb0fe5d1db75540a4805cb58b6"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.743 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed', u'name': u'vumrao'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "vumrao", "ovpn": "vumrao@redhat TMNqzMvbJS8Va/8nT9QUQw ab386c2bd7c6796d5413e4d841a16dda2504cca6d95df831a652a30d2e5655ed"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.753 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57', u'name': u'kmroz'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kmroz", "ovpn": "kmroz@suse /JbrIs2mKL5exdmcDnhRgg db4d19ab99c7174429d5ae7b6ca3cf4e04e9bf7810e1826d90f4627643628d57"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.770 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736', u'name': u'henrix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "henrix", "ovpn": "henrix@hermes iPPDBfnLzP5Pe5FuTcJBmw b26aefb8a61451066f984e074f708ea9ca6b2c5d7cca35996c08b0b2bb2c2736"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.771 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26', u'name': u'pbs1108'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pbs1108", "ovpn": "pbs1108@bspark_cli0 nSCINdeTLTLEO5JP/GIwRQ 76372ad6f7ad731556ff13605c3729eacaf59dcf7f9ac82dd9a8501bd95d3b26"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.782 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315', u'name': u'clacroix'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "clacroix", "ovpn": "clacroix@redhat.com ZGY9sgvjT0BuJRi9zrULzg c3311aba4025aa42cd78c999dcee4e2c16415a3ac44ac8c95b77838459ef3315"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.783 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006', u'name': u'epuertat'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "epuertat", "ovpn": "epuertat@private BnUoirwxGRWXtLxulJU5xA da2cfc4995bed82ef85db3633edad0a7eb2c32ba559a48259b10be94a8fdf006"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.793 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461', u'name': u'tdehler'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "tdehler", "ovpn": "tdehler@think 7e0WC4Vh86XWZviZ9WBMgw 4dc8477db6e4f40312e6b2b9db293dc009e49e518015ace20431c0fb69025461"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.804 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305', u'name': u'laura'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "laura", "ovpn": "laura@flab 2DGHIAxD39eNKOPkn3M67w ab1ae304abed3824a68b5c0ecf4f92fca76a4f8b9fcbcc0ca43388a85b7f9305"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.815 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n', u'name': u'adamyanova'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "adamyanova", "ovpn": "adamyanova@centos k2nfk8hmGW77nys3R/JkOg 7695b95c2f157d60622e0b0b7ab06fa2cb18661a190d839f7ea587bc44aa0e3c\nadamyanova@ubuntu FmlKgjVzPUxfNDnHeU9vLQ ef7d5524863dfa0787fc5e249873c1a5ea58e7fd5aee27e1d1d33d6f87388a2d\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.828 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce', u'name': u'yaarit'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yaarit", "ovpn": "yaarit@centos TBuWjkAsj1GB/V9eWc/R1Q 7bd86a857dec48dc25850ecf0c00486d9a89c2ff5f88b2f28c3e36bdeb139fce"}, "skip_reason": "Conditional result was False"} skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3', u'name': u'rpavani1998'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rpavani1998", "ovpn": "admin1@rajula 31GbDo9d1YnW5BQ8u3utvw a2da13cb840f848846023c85442ba7bcce97dc186056a0ecc036a220d7eb7fc3"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.839 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4', u'name': u'rishabh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rishabh", "ovpn": "rishabh@p50 zdJ4XsBdVugwMrqJOSBi3Q c78bb28ba5cf2bf9c8edb80fe57814d60cd2ffdbd874cf9a271e5adf171bb0c4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.849 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2', u'name': u'skrah'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "skrah", "ovpn": "skrah@thinkpad IboPmnUdsLqqRXlHQ1RT5w bf85db9e916dceaf84a1e6ea33c59eb4adb424cb4e727ce0a903a3498b799ed2"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.852 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8', u'name': u'smanjara'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "smanjara", "ovpn": "smanjara@fedora 5oorMoVYD3sT0nmOTBDh9w 83be007f68694c9463ef46e4ce223221d639d78f11d5b68449598de77e8e0ce8"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.860 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41', u'name': u'bengland'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "bengland", "ovpn": "bengland@bene-laptop ud1gWgoNggJTS7LQtPTZTA d3ebd084ce385cb450ce2f83c02dc66a1637dedbc7a8b191dab68acfc935af41"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.875 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d', u'name': u'pnawracay'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "pnawracay", "ovpn": "pnawracay@flab LIurEXLa7xbr+jzf2wRJVg 04062ba385602b385fd17f14de3a0cad83c685b8078fd2f18cc9ad77a4f4762d"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.877 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8', u'name': u'alfonsomthd'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "alfonsomthd", "ovpn": "alfonsomthd@localhost eUvx+7UEx5IYGmS0lNIscQ 2f1bfd4874280b9f525a46e93e767504df80a9b09a83a2fea387dcd6e34bc0f8"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.887 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e', u'name': u'oliveiradan'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "oliveiradan", "ovpn": "oliveiradan@opensuse-dev twDqMYwYsdYmbgyCpUnElw ec3ee80ddc747c3ca5e1455a122279f8e1e642c5c09aa9c2ca7fec142f55089e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.898 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba', u'name': u'swagner'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "swagner", "ovpn": "swagner@ubuntu 64V1h0Se0FmBQNH7KLibbQ ad7c91e9e2f7f3999492d5e41fbbc993327d37929bd09606227367d75e5556ba"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.899 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi081.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-11-25T09:09:02.901 INFO:teuthology.task.ansible.out: TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e', u'name': u'yuvalif'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "yuvalif", "ovpn": "ylifshit@localhost dyc2NU2pMz8NF8/uR2kMxA 3a6f1f9e55b5116f74d01ffbabdc339054088d257a16cf9fafcfe05b27fa678e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.911 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852', u'name': u'kkeithle'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kkeithle", "ovpn": "kkeithle@kkeithle.usersys.redhat.com FPnVevv1sp5hlWoJeDCe/g e5a1fa7ccf678b91ed570983d5420c98f109d507442c8e4dcd50803e0d71c852"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.922 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130', u'name': u'emmericp'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "emmericp", "ovpn": "paul.emmerich@croit.io RN4hOorvA7irUg/3ViM9KQ 3bd06194186d2624cadf255fa1c38ddf7dded0a6d83dc6001cd55fcc0a899130"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.923 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0', u'name': u'mchangir'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "mchangir", "ovpn": "mchangir@indraprastha 1p71eDTdLMzkySFg0d5qEg bc4fda0ac9ef5cfbfab0cd9e5ed956c75b6280a04043c90c288e62b416b42da0"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.934 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286', u'name': u'sidharthanup'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sidharthanup", "ovpn": "sidharthanup@strawberryfields IZq91vqA+RG1Rtn3JFZb6Q a2873481cac9b8b4a0bd8bebe0248b3dccb370dd18b56a4dae713ca1fb0c4286"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.945 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31', u'name': u'varsha'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "varsha", "ovpn": "varsha@local AGrJViO6wZF253aFeh/r5w f1248029ccdc45fd6e8fddb09ab3f8aa01123ab69cc6f3f11273528ce33e5a31"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.956 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n', u'name': u'sjust'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sjust", "ovpn": "sjust@pondermatic sCd6606QpID1PnHn0AxFag 46da7d4c77cb1238f83a34f99e774707110f997d88f3a0fd240aac9b7b7bbc85\nsjust@rex004 YS0in6YtQJHx5aUo7ZHi8Q bdd5977d05171a365539b19fae283ec2e3c7389516664692b9bbbaf98c7b61f4\nsjust@office w19UilyC/xu7uCzv0DnWRg ab20efc114b769bf4c2cf313eb30db09c2e2f8234992f120cfc3d1b8b347ed3c\nsam@deepthought 44sCi+GEfY0zjKo5M/4FiQ ed1eedd14ca68116a2000477fa078f8f736d0a15640723c32204bb30f14cb888\n"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.968 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f', u'name': u'ideepika'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ideepika", "ovpn": "deepika@asus hHsL1xugca0LzY52gKqfqQ 312e0f2680f72d9459c707fcd0ccfb777617f00017f0511839e9b7e3167d590f"}, "skip_reason": "Conditional result was False"} skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46', u'name': u'gsalomon'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "gsalomon", "ovpn": "gsalomon@home BrlCVxStNlEZGf5yYYAz1w 4b0c8d5b57dae1328c16d3017b59b632ccdfebe4135209fa97748c70ff00cc46"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.978 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de', u'name': u'soumyakoduri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "soumyakoduri", "ovpn": "skoduri@rgw_vm fAskaxmJYtm4TDcModIkrQ 884b57c3b5d56493da361dc9f1cc4e06e766628fcb0f916090f2096edc5ce7de"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:02.989 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a', u'name': u'kyr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "kyr", "ovpn": "kyr@suse Xu4+Yi98Il4ETOAav6okqA 03bc46e5fac6346cd82ff681f756860f98e0c61168633ce23325efde11a1964a"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.001 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849', u'name': u'sseshasa'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "sseshasa", "ovpn": "sseshasa@thinkpad jyB1pr0I3qsDknkTnJTMjg 72ac1456e344c22fd940d0ba0e035aa3819ef7cd3891e53308aa92ba2dec8849"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.012 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189', u'name': u'rfriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rfriedma", "ovpn": "rfriedma@rflap.redhat.com 5+OUPoyz8K0M0kcymdQOjA 40ce705001f31d7156c965228938cd4b02ae1a2c43dac1bbcd1b538e70312189"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.015 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b', u'name': u'lmb'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "lmb", "ovpn": "lmb@hermes LMS8kAikL0iqw2S6IbXa3Q f57a493b31e7ed02a2563dd4295278d4842dc698b4c635d011a8d2b4b1fd5c2b"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.024 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc', u'name': u'ksirivad'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ksirivad", "ovpn": "ksirivad@ksirivad prZzE08FqnY6hFtGk0Q6XQ 2ef0878b0050cf28775813fe0991f9a746c07e61920280ce29ee69088eda5efc"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.153 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4', u'name': u'ofriedma'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ofriedma", "ovpn": "ofriedma@myhost IIifKuFYfX5EqO3mhV8lAg 82cb23a98361f3624b2f1f4e0f694f9439ae7735b171ba5e3bb5f0744f5ddfc4"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.164 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52', u'name': u'h4iresis'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "h4iresis", "ovpn": "h4iresis@fedora NyMdx+dAfmujNSDB63kI+g bcf6c36594585e0fbf9f037bf1a13fe63db4b82a52ff1349349fac748b5c4e52"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.165 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5', u'name': u'shyukri'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "shyukri", "ovpn": "shyukri@t480 KEbvWFIOWqkHsLgffi6GMQ fcd405d0cf6b2c7ba53397c4ed1156e852317fb02bcd6f0e98e5eb3c1c3aa4b5"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.182 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e', u'name': u'scarvalhojr'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "scarvalhojr", "ovpn": "scarvalhojr@milestones utvFbB+X1G75M589jLfFXQ 9c62bf416c9204b25028ba0e5bd45f754c4aa8f84a85b9d5ded02aa83f03e82e"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.189 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b', u'name': u'rakesh'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rakesh", "ovpn": "rakesh@fedora CXsWP7Wzu2C+bTN6mIeMiw 2f4f492cd085f1f46d1d5018aeb7705ba855c4481e230a8188b8f5ea9994b91b"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.204 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486', u'name': u'ryg-'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "ryg-", "ovpn": "ryg@wombat 7WbRbRXPcrQ2++Bww59JKg 1186db58c70662d55cae6d75e54022d1392708d3cf742b10ce5ff9d67dd8486"}, "skip_reason": "Conditional result was False"} skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3', u'name': u'prsrivas'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "prsrivas", "ovpn": "prsrivas@t480 dvGwTZM2FivQPrMGmkY53g 80afabbb654af66ec01463d0e0425be5643eaca10bbf756f2b8313645d2b8da3"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.225 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42', u'name': u'zdover'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "zdover", "ovpn": "zdover@slim_thinkpad H/78rBqDIJ5AAjMT6sTmKg 49fb166c553d14549491280458269f987bea97295dbf525ffb7c35336175ee42"}, "skip_reason": "Conditional result was False"} TASK [users : Remove revoked users] ******************************************** ok: [smithi081.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} TASK [users : Update authorized_keys for each user with literal keys] ********** skipping: [smithi153.front.sepia.ceph.com] => (item={u'ovpn': u'rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b', u'name': u'rraja'}) => {"ansible_loop_var": "item", "changed": false, "item": {"name": "rraja", "ovpn": "rraja@thinkpadp50 8S3Lez75woSBdYiLqQxf2g c9d1666dfc7a3b762700cd250f775a506c397b0f0e6a86289ba555a3546e532b"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.288 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:03 +0000 (0:00:01.423) 0:01:34.669 ******* 2019-11-25T09:09:03.317 INFO:teuthology.task.ansible.out: 2019-11-25T09:09:03.319 INFO:teuthology.task.ansible.out:TASK [users : Filter the revoked_users list] *********************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:03.361 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:03 +0000 (0:00:00.071) 0:01:34.741 ******* 2019-11-25T09:09:03.547 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi081.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-11-25T09:09:03.764 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=zcerza) => {"ansible_loop_var": "item", "changed": false, "item": "zcerza", "name": "zcerza", "state": "absent"} 2019-11-25T09:09:03.870 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-11-25T09:09:04.090 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=aschoen) => {"ansible_loop_var": "item", "changed": false, "item": "aschoen", "name": "aschoen", "state": "absent"} 2019-11-25T09:09:04.198 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-11-25T09:09:04.395 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=andrew) => {"ansible_loop_var": "item", "changed": false, "item": "andrew", "name": "andrew", "state": "absent"} 2019-11-25T09:09:04.511 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-11-25T09:09:04.719 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=sweil) => {"ansible_loop_var": "item", "changed": false, "item": "sweil", "name": "sweil", "state": "absent"} 2019-11-25T09:09:04.836 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-11-25T09:09:05.045 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=brad) => {"ansible_loop_var": "item", "changed": false, "item": "brad", "name": "brad", "state": "absent"} 2019-11-25T09:09:05.149 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-11-25T09:09:05.381 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=kefu) => {"ansible_loop_var": "item", "changed": false, "item": "kefu", "name": "kefu", "state": "absent"} 2019-11-25T09:09:05.487 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-11-25T09:09:05.721 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=shylesh) => {"ansible_loop_var": "item", "changed": false, "item": "shylesh", "name": "shylesh", "state": "absent"} 2019-11-25T09:09:05.780 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-11-25T09:09:06.049 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=trhoden) => {"ansible_loop_var": "item", "changed": false, "item": "trhoden", "name": "trhoden", "state": "absent"} 2019-11-25T09:09:06.103 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-11-25T09:09:06.354 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=nishtha) => {"ansible_loop_var": "item", "changed": false, "item": "nishtha", "name": "nishtha", "state": "absent"} 2019-11-25T09:09:06.428 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-11-25T09:09:06.695 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=yguang) => {"ansible_loop_var": "item", "changed": false, "item": "yguang", "name": "yguang", "state": "absent"} 2019-11-25T09:09:06.768 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-11-25T09:09:06.997 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=sdieffen) => {"ansible_loop_var": "item", "changed": false, "item": "sdieffen", "name": "sdieffen", "state": "absent"} 2019-11-25T09:09:07.092 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-11-25T09:09:07.343 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=brian) => {"ansible_loop_var": "item", "changed": false, "item": "brian", "name": "brian", "state": "absent"} 2019-11-25T09:09:07.397 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-11-25T09:09:07.649 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=pmcgarry) => {"ansible_loop_var": "item", "changed": false, "item": "pmcgarry", "name": "pmcgarry", "state": "absent"} 2019-11-25T09:09:07.733 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-11-25T09:09:07.992 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=karnan) => {"ansible_loop_var": "item", "changed": false, "item": "karnan", "name": "karnan", "state": "absent"} 2019-11-25T09:09:08.065 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-11-25T09:09:08.314 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=ryneli) => {"ansible_loop_var": "item", "changed": false, "item": "ryneli", "name": "ryneli", "state": "absent"} 2019-11-25T09:09:08.377 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-11-25T09:09:08.656 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=dlambrig) => {"ansible_loop_var": "item", "changed": false, "item": "dlambrig", "name": "dlambrig", "state": "absent"} 2019-11-25T09:09:08.719 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-11-25T09:09:08.959 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=icolle) => {"ansible_loop_var": "item", "changed": false, "item": "icolle", "name": "icolle", "state": "absent"} 2019-11-25T09:09:09.043 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-11-25T09:09:09.293 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=soumya) => {"ansible_loop_var": "item", "changed": false, "item": "soumya", "name": "soumya", "state": "absent"} 2019-11-25T09:09:09.376 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-11-25T09:09:09.626 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=jspray) => {"ansible_loop_var": "item", "changed": false, "item": "jspray", "name": "jspray", "state": "absent"} 2019-11-25T09:09:09.689 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-11-25T09:09:09.969 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=loic) => {"ansible_loop_var": "item", "changed": false, "item": "loic", "name": "loic", "state": "absent"} 2019-11-25T09:09:10.011 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-11-25T09:09:10.312 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=erwan) => {"ansible_loop_var": "item", "changed": false, "item": "erwan", "name": "erwan", "state": "absent"} 2019-11-25T09:09:10.344 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-11-25T09:09:10.634 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=nlevine) => {"ansible_loop_var": "item", "changed": false, "item": "nlevine", "name": "nlevine", "state": "absent"} 2019-11-25T09:09:10.645 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-11-25T09:09:10.984 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=tbrekke) => {"ansible_loop_var": "item", "changed": false, "item": "tbrekke", "name": "tbrekke", "state": "absent"} 2019-11-25T09:09:10.989 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-11-25T09:09:11.055 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:11 +0000 (0:00:07.694) 0:01:42.436 ******* 2019-11-25T09:09:11.308 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=taco) => {"ansible_loop_var": "item", "changed": false, "item": "taco", "name": "taco", "state": "absent"} 2019-11-25T09:09:11.622 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=louis) => {"ansible_loop_var": "item", "changed": false, "item": "louis", "name": "louis", "state": "absent"} 2019-11-25T09:09:11.967 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=amarango) => {"ansible_loop_var": "item", "changed": false, "item": "amarango", "name": "amarango", "state": "absent"} 2019-11-25T09:09:12.118 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-11-25T09:09:12.120 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "checksum": "50650ef8f5fa65d26a4b4e7fa94b1571b873d967", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "f08a845f35203e506ea1deb151ba8443", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1574672951.09-127744289184055/source", "state": "file", "uid": 0} 2019-11-25T09:09:12.157 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:12 +0000 (0:00:01.101) 0:01:43.537 ******* 2019-11-25T09:09:12.194 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** ok: [smithi081.front.sepia.ceph.com] => { "msg": "Host smithi081.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-11-25T09:09:12.237 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:12 +0000 (0:00:00.076) 0:01:43.614 ******* 2019-11-25T09:09:12.271 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** ok: [smithi081.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-11-25T09:09:12.326 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:12 +0000 (0:00:00.087) 0:01:43.701 ******* TASK [users : Remove revoked users] ******************************************** ok: [smithi153.front.sepia.ceph.com] => (item=oobe) => {"ansible_loop_var": "item", "changed": false, "item": "oobe", "name": "oobe", "state": "absent"} 2019-11-25T09:09:12.348 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:12.390 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:12 +0000 (0:00:00.068) 0:01:43.769 ******* 2019-11-25T09:09:12.637 INFO:teuthology.task.ansible.out: TASK [users : Remove revoked users] ******************************************** ok: [smithi153.front.sepia.ceph.com] => (item=rturk) => {"ansible_loop_var": "item", "changed": false, "item": "rturk", "name": "rturk", "state": "absent"} 2019-11-25T09:09:12.703 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:12 +0000 (0:00:00.310) 0:01:44.080 ******* 2019-11-25T09:09:12.822 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.113387", "end": "2019-11-25 09:09:12.779930", "rc": 0, "start": "2019-11-25 09:09:12.666543", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-11-25T09:09:12.864 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:12 +0000 (0:00:00.161) 0:01:44.241 ******* 2019-11-25T09:09:12.900 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:12.948 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:12 +0000 (0:00:00.085) 0:01:44.326 ******* 2019-11-25T09:09:12.992 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.035 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.084) 0:01:44.412 ******* 2019-11-25T09:09:13.057 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-11-25T09:09:13.059 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.103 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.067) 0:01:44.479 ******* 2019-11-25T09:09:13.125 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.166 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.067) 0:01:44.547 ******* 2019-11-25T09:09:13.194 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.234 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.066) 0:01:44.614 ******* 2019-11-25T09:09:13.262 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.300 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.066) 0:01:44.680 ******* 2019-11-25T09:09:13.339 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** skipping: [smithi081.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi081.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:13.376 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.076) 0:01:44.757 ******* 2019-11-25T09:09:13.404 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-11-25T09:09:13.405 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.446 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.069) 0:01:44.827 ******* 2019-11-25T09:09:13.474 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** 2019-11-25T09:09:13.476 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.531 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.084) 0:01:44.911 ******* 2019-11-25T09:09:13.557 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-11-25T09:09:13.557 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:13.595 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.063) 0:01:44.975 ******* 2019-11-25T09:09:13.621 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-11-25T09:09:13.623 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.663 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.066) 0:01:45.043 ******* 2019-11-25T09:09:13.689 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-11-25T09:09:13.690 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:13.730 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.067) 0:01:45.111 ******* 2019-11-25T09:09:13.756 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-11-25T09:09:13.758 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.801 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.070) 0:01:45.182 ******* 2019-11-25T09:09:13.807 INFO:teuthology.task.ansible.out: TASK [users : Write /keys-repo-sha1 sentinel file] ***************************** 2019-11-25T09:09:13.808 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "checksum": "50650ef8f5fa65d26a4b4e7fa94b1571b873d967", "dest": "/keys-repo-sha1", "gid": 0, "group": "root", "md5sum": "f08a845f35203e506ea1deb151ba8443", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:etc_runtime_t:s0", "size": 40, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1574672952.73-216013186550465/source", "state": "file", "uid": 0} 2019-11-25T09:09:13.846 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.044) 0:01:45.227 ******* 2019-11-25T09:09:13.851 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-11-25T09:09:13.852 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.894 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.047) 0:01:45.275 ******* 2019-11-25T09:09:13.899 INFO:teuthology.task.ansible.out: TASK [common : Log the OS name, version and release] *************************** 2019-11-25T09:09:13.901 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => { "msg": "Host smithi153.front.sepia.ceph.com is running CentOS 7.6 (Core)" } 2019-11-25T09:09:13.939 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.044) 0:01:45.319 ******* 2019-11-25T09:09:13.944 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-11-25T09:09:13.945 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:13.984 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:13 +0000 (0:00:00.045) 0:01:45.365 ******* 2019-11-25T09:09:13.989 INFO:teuthology.task.ansible.out: TASK [common : Including major version specific variables.] ******************** 2019-11-25T09:09:13.990 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/empty.yml"} 2019-11-25T09:09:14.029 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.044) 0:01:45.410 ******* 2019-11-25T09:09:14.035 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-11-25T09:09:14.035 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.075 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.045) 0:01:45.456 ******* 2019-11-25T09:09:14.080 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 6)] *********************** 2019-11-25T09:09:14.081 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.121 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.046) 0:01:45.502 ******* 2019-11-25T09:09:14.123 INFO:teuthology.task.ansible.out: 2019-11-25T09:09:14.127 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-11-25T09:09:14.127 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:14.170 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.048) 0:01:45.551 ******* 2019-11-25T09:09:14.196 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ 2019-11-25T09:09:14.198 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.246 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.075) 0:01:45.627 ******* 2019-11-25T09:09:14.276 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.317 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.070) 0:01:45.698 ******* 2019-11-25T09:09:14.344 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.381 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.063) 0:01:45.762 ******* 2019-11-25T09:09:14.407 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-11-25T09:09:14.409 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.449 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.067) 0:01:45.829 ******* 2019-11-25T09:09:14.491 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-11-25T09:09:14.493 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.528 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.079) 0:01:45.909 ******* 2019-11-25T09:09:14.555 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone (RHEL/CentOS 7)] *********************** 2019-11-25T09:09:14.556 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "cmd": "timedatectl | grep -E \"Time ?zone\" | sed -e \"s/.*: \\(.*\\) (.*/\\1/\"", "delta": "0:00:00.117587", "end": "2019-11-25 09:09:14.520312", "rc": 0, "start": "2019-11-25 09:09:14.402725", "stderr": "", "stderr_lines": [], "stdout": "Etc/UTC", "stdout_lines": ["Etc/UTC"]} 2019-11-25T09:09:14.591 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.063) 0:01:45.972 ******* 2019-11-25T09:09:14.605 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.639 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.047) 0:01:46.019 ******* 2019-11-25T09:09:14.647 INFO:teuthology.task.ansible.out: TASK [common : Reboot RHEL7 to workaround systemd bug] ************************* 2019-11-25T09:09:14.648 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.707 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.068) 0:01:46.088 ******* 2019-11-25T09:09:14.712 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** 2019-11-25T09:09:14.713 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:14.754 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.047) 0:01:46.135 ******* 2019-11-25T09:09:14.760 INFO:teuthology.task.ansible.out: TASK [common : Wait for reboot in case of systemd workaround] ****************** 2019-11-25T09:09:14.762 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.801 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.046) 0:01:46.181 ******* 2019-11-25T09:09:14.806 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-11-25T09:09:14.844 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.043) 0:01:46.224 ******* 2019-11-25T09:09:14.849 INFO:teuthology.task.ansible.out: TASK [common : Set /etc/localtime (RHEL/CentOS 6)] ***************************** 2019-11-25T09:09:14.851 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.890 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.046) 0:01:46.271 ******* 2019-11-25T09:09:14.896 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-11-25T09:09:14.897 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.935 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.044) 0:01:46.315 ******* 2019-11-25T09:09:14.940 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone (RHEL/CentOS >= 7)] **************************** 2019-11-25T09:09:14.942 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:14.979 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:14 +0000 (0:00:00.044) 0:01:46.360 ******* 2019-11-25T09:09:15.015 INFO:teuthology.task.ansible.out: TASK [common : Install yum on Fedora 22 and later] ***************************** 2019-11-25T09:09:15.017 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.056 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.076) 0:01:46.436 ******* 2019-11-25T09:09:15.085 INFO:teuthology.task.ansible.out: TASK [common : Set entitlements_path] ****************************************** 2019-11-25T09:09:15.086 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.125 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.069) 0:01:46.505 ******* 2019-11-25T09:09:15.162 INFO:teuthology.task.ansible.out: TASK [common : Include Red Hat encrypted variables.] *************************** 2019-11-25T09:09:15.164 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=None) => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} skipping: [smithi153.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:15.200 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.075) 0:01:46.581 ******* 2019-11-25T09:09:15.226 INFO:teuthology.task.ansible.out: TASK [common : Set have_entitlements] ****************************************** 2019-11-25T09:09:15.228 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.264 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.063) 0:01:46.645 ******* 2019-11-25T09:09:15.268 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-11-25T09:09:15.269 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:09:15.309 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.044) 0:01:46.690 ******* 2019-11-25T09:09:15.330 INFO:teuthology.task.ansible.out: TASK [common : Install CA Cert from Satellite Server] ************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.356 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.045) 0:01:46.735 ******* 2019-11-25T09:09:15.391 INFO:teuthology.task.ansible.out: TASK [common : Determine if node is registered with subscription-manager.] ***** 2019-11-25T09:09:15.394 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:15.431 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.076) 0:01:46.812 ******* 2019-11-25T09:09:15.457 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we're already registered] **************** 2019-11-25T09:09:15.459 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.497 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.065) 0:01:46.878 ******* 2019-11-25T09:09:15.533 INFO:teuthology.task.ansible.out: TASK [common : Register with subscription-manager.] **************************** 2019-11-25T09:09:15.535 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:15.570 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.073) 0:01:46.951 ******* 2019-11-25T09:09:15.596 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered if we just registered] ********************** 2019-11-25T09:09:15.597 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.636 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.062) 0:01:47.013 ******* 2019-11-25T09:09:15.658 INFO:teuthology.task.ansible.out: TASK [common : List CDN releases available to system] ************************** 2019-11-25T09:09:15.660 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.695 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.062) 0:01:47.076 ******* 2019-11-25T09:09:15.721 INFO:teuthology.task.ansible.out: TASK [common : Unregister Beta/Alpha/RC system with subscription-manager] ****** 2019-11-25T09:09:15.722 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.758 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.062) 0:01:47.138 ******* 2019-11-25T09:09:15.785 INFO:teuthology.task.ansible.out: TASK [common : Set rhsm_registered to false if Beta/Alpha/RC release] ********** 2019-11-25T09:09:15.787 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.825 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.067) 0:01:47.206 ******* 2019-11-25T09:09:15.853 INFO:teuthology.task.ansible.out: TASK [common : Configure internal beta repos in /etc/yum.repos.d/] ************* 2019-11-25T09:09:15.854 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:15.891 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.065) 0:01:47.272 ******* 2019-11-25T09:09:15.918 INFO:teuthology.task.ansible.out: TASK [common : Clean yum cache] ************************************************ skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:15.956 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:15 +0000 (0:00:00.064) 0:01:47.337 ******* 2019-11-25T09:09:15.982 INFO:teuthology.task.ansible.out: TASK [common : Get list of enabled RHSM repos] ********************************* 2019-11-25T09:09:15.984 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:16.027 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.069) 0:01:47.407 ******* 2019-11-25T09:09:16.063 INFO:teuthology.task.ansible.out: TASK [common : Store list of enabled repos] ************************************ 2019-11-25T09:09:16.064 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:16.110 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.084) 0:01:47.491 ******* 2019-11-25T09:09:16.137 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos false if entitlements are missing or if we unregistered] *** 2019-11-25T09:09:16.138 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:16.179 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.069) 0:01:47.560 ******* 2019-11-25T09:09:16.216 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if rhsm_repos differs from repo_list] **** 2019-11-25T09:09:16.221 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:16.255 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.075) 0:01:47.636 ******* 2019-11-25T09:09:16.281 INFO:teuthology.task.ansible.out: TASK [common : Set replace_repos true if newly-subscribed] ********************* 2019-11-25T09:09:16.283 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:16.322 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.066) 0:01:47.703 ******* 2019-11-25T09:09:16.349 INFO:teuthology.task.ansible.out: TASK [common : Disable all rhsm repos] ***************************************** skipping: [smithi153.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": false} 2019-11-25T09:09:16.396 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.074) 0:01:47.777 ******* 2019-11-25T09:09:16.429 INFO:teuthology.task.ansible.out: TASK [common : Enable necessary rhsm repos] ************************************ 2019-11-25T09:09:16.473 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.075) 0:01:47.853 ******* 2019-11-25T09:09:16.500 INFO:teuthology.task.ansible.out: TASK [common : Remove old apt-mirror repository definition.] ******************* 2019-11-25T09:09:16.502 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:16.556 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.083) 0:01:47.937 ******* 2019-11-25T09:09:16.562 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-11-25T09:09:16.563 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-11-25T09:09:16.898 INFO:teuthology.task.ansible.out: TASK [common : Increase the yum timeout.] ************************************** 2019-11-25T09:09:16.900 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:09:16.942 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:16 +0000 (0:00:00.382) 0:01:48.319 ******* 2019-11-25T09:09:17.408 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** ok: [smithi081.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-11-25T09:09:17.456 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:17 +0000 (0:00:00.510) 0:01:48.830 ******* 2019-11-25T09:09:17.883 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "1d6dcd385d1b7c9c2b757c62b841c887e90d7f66", "dest": "/etc/yum.repos.d/epel.repo", "gid": 0, "group": "root", "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 293, "state": "file", "uid": 0} 2019-11-25T09:09:18.491 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-11-25T09:09:18.493 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1574672957.49-263602405842678/source", "state": "file", "uid": 0} 2019-11-25T09:09:18.855 INFO:teuthology.task.ansible.out: TASK [common : Configure epel repos in /etc/yum.repos.d/] ********************** 2019-11-25T09:09:18.856 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "d9691278f687ee76cfe8a3de60fcbd4f1f79a8c6", "dest": "/etc/yum.repos.d/epel-testing.repo", "gid": 0, "group": "root", "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/epel-testing.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 327, "state": "file", "uid": 0} 2019-11-25T09:09:18.899 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:18 +0000 (0:00:01.446) 0:01:50.276 ******* 2019-11-25T09:09:19.463 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* 2019-11-25T09:09:19.464 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1574672958.51-167250385760488/source", "state": "file", "uid": 0} 2019-11-25T09:09:19.502 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:19 +0000 (0:00:00.606) 0:01:50.883 ******* 2019-11-25T09:09:19.985 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item=epel-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "73c387451a4fa299c9cb53e25b182ae50e8b8563", "dest": "/etc/yum.repos.d/epel-mirrorlist", "gid": 0, "group": "root", "item": "epel-mirrorlist", "md5sum": "6467980f3cbb5aa63a3cfe239626f53e", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 418, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1574672958.93-88934336075378/source", "state": "file", "uid": 0} 2019-11-25T09:09:20.092 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-11-25T09:09:20.094 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-11-25T09:09:20.409 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-11-25T09:09:20.411 INFO:teuthology.task.ansible.out: [WARNING]: Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually 2019-11-25T09:09:20.451 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:20 +0000 (0:00:00.949) 0:01:51.832 ******* 2019-11-25T09:09:21.062 INFO:teuthology.task.ansible.out: TASK [common : Configure local epel mirrorlists] ******************************* changed: [smithi153.front.sepia.ceph.com] => (item=epel-testing-mirrorlist) => {"ansible_loop_var": "item", "changed": true, "checksum": "9d25e11bf71dfd21a39ace708c03408897a4f263", "dest": "/etc/yum.repos.d/epel-testing-mirrorlist", "gid": 0, "group": "root", "item": "epel-testing-mirrorlist", "md5sum": "0848bb8607c5b0c5706d8e1930fa7d55", "mode": "0644", "owner": "root", "secontext": "system_u:object_r:system_conf_t:s0", "size": 474, "src": "/home/ubuntu/.ansible/tmp/ansible-tmp-1574672960.0-9205635742/source", "state": "file", "uid": 0} 2019-11-25T09:09:21.100 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:21 +0000 (0:00:00.648) 0:01:52.481 ******* 2019-11-25T09:09:21.459 INFO:teuthology.task.ansible.out: TASK [common : Temporarily comment mirrorlist so we can update yum cache] ****** 2019-11-25T09:09:21.460 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made", "warnings": ["Module remote_tmp /root/.ansible/tmp did not exist and was created with a mode of 0700, this may cause issues when running as another user. To avoid this, create the remote_tmp dir with the correct permissions manually"]} 2019-11-25T09:09:21.720 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-11-25T09:09:21.763 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:21 +0000 (0:00:00.662) 0:01:53.144 ******* 2019-11-25T09:09:32.958 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** 2019-11-25T09:09:32.960 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the file module with state=absent rather than running 'rm'. If you need to use command because file is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.183758", "end": "2019-11-25 09:09:32.925513", "rc": 0, "start": "2019-11-25 09:09:20.741755", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: packages.oit.ncsu.edu\n * extras: mirror.jaleco.com\n * updates: mirror.atlanticmetro.net\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: packages.oit.ncsu.edu", " * extras: mirror.jaleco.com", " * updates: mirror.atlanticmetro.net", "Metadata Cache Created"]} 2019-11-25T09:09:33.002 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:32 +0000 (0:00:11.239) 0:02:04.383 ******* 2019-11-25T09:09:33.348 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-11-25T09:09:33.671 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-11-25T09:09:33.709 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:33 +0000 (0:00:00.707) 0:02:05.090 ******* 2019-11-25T09:09:33.737 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-11-25T09:09:33.739 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:33.776 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:33 +0000 (0:00:00.066) 0:02:05.157 ******* 2019-11-25T09:09:33.805 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** 2019-11-25T09:09:33.806 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:33.845 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:33 +0000 (0:00:00.068) 0:02:05.225 ******* 2019-11-25T09:09:33.882 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:33.926 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:33 +0000 (0:00:00.081) 0:02:05.307 ******* 2019-11-25T09:09:33.953 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-11-25T09:09:33.954 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:33.989 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:33 +0000 (0:00:00.063) 0:02:05.370 ******* 2019-11-25T09:09:34.016 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** 2019-11-25T09:09:34.018 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:34.054 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:34 +0000 (0:00:00.064) 0:02:05.434 ******* 2019-11-25T09:09:34.081 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ 2019-11-25T09:09:34.082 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:34.118 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:34 +0000 (0:00:00.064) 0:02:05.499 ******* 2019-11-25T09:09:34.145 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* 2019-11-25T09:09:34.146 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:34.182 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:34 +0000 (0:00:00.063) 0:02:05.563 ******* 2019-11-25T09:09:34.208 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-11-25T09:09:34.210 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:34.245 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:34 +0000 (0:00:00.063) 0:02:05.626 ******* 2019-11-25T09:09:34.890 INFO:teuthology.task.ansible.out: TASK [common : Update yum cache] *********************************************** 2019-11-25T09:09:34.891 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": "rm -rf /var/cache/yum/*; yum clean all; yum makecache", "delta": "0:00:12.802061", "end": "2019-11-25 09:09:34.860723", "rc": 0, "start": "2019-11-25 09:09:22.058662", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nCleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates\nLoaded plugins: fastestmirror, langpacks, priorities\nDetermining fastest mirrors\n * base: packages.oit.ncsu.edu\n * extras: mirror.jaleco.com\n * updates: mirror.trouble-free.net\nMetadata Cache Created", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Cleaning repos: base centos7-fcgi-ceph epel extras lab-extras updates", "Loaded plugins: fastestmirror, langpacks, priorities", "Determining fastest mirrors", " * base: packages.oit.ncsu.edu", " * extras: mirror.jaleco.com", " * updates: mirror.trouble-free.net", "Metadata Cache Created"]} 2019-11-25T09:09:34.946 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:34 +0000 (0:00:00.700) 0:02:06.327 ******* 2019-11-25T09:09:35.412 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-11-25T09:09:35.412 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-mirrorlist', u'name': u'Extra Packages for Enterprise Linux', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': u'1', u'baseurl': u'http://dl.fedoraproject.org/pub/epel/7/$basearch'}, 'key': u'epel'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/7/$basearch", "enabled": "1", "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-mirrorlist", "name": "Extra Packages for Enterprise Linux"}}, "msg": "1 replacements made"} 2019-11-25T09:09:35.531 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-11-25T09:09:35.533 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.988880", "end": "2019-11-25 09:09:35.500601", "rc": 0, "start": "2019-11-25 09:09:34.511721", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:09:35.568 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:35 +0000 (0:00:00.622) 0:02:06.949 ******* 2019-11-25T09:09:35.729 INFO:teuthology.task.ansible.out: TASK [common : Uncomment mirrorlist] ******************************************* 2019-11-25T09:09:35.730 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'mirrorlist': u'file:///etc/yum.repos.d/epel-testing-mirrorlist', u'name': u'Extra Packages for Enterprise Linux - Testing', u'failovermethod': u'priority', u'gpgcheck': 0, u'enabled': 0, u'baseurl': u'http://dl.fedoraproject.org/pub/epel/testing/7/$basearch'}, 'key': u'epel-testing'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "epel-testing", "value": {"baseurl": "http://dl.fedoraproject.org/pub/epel/testing/7/$basearch", "enabled": 0, "failovermethod": "priority", "gpgcheck": 0, "mirrorlist": "file:///etc/yum.repos.d/epel-testing-mirrorlist", "name": "Extra Packages for Enterprise Linux - Testing"}}, "msg": "1 replacements made"} 2019-11-25T09:09:35.870 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:35 +0000 (0:00:00.301) 0:02:07.250 ******* 2019-11-25T09:09:35.896 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** 2019-11-25T09:09:35.898 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:35.935 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:35 +0000 (0:00:00.065) 0:02:07.316 ******* 2019-11-25T09:09:35.977 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone in /etc/timezone] ****************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.010 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.075) 0:02:07.391 ******* 2019-11-25T09:09:36.047 INFO:teuthology.task.ansible.out: TASK [common : Inform the OS of the new timezone] ****************************** 2019-11-25T09:09:36.049 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.085 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.074) 0:02:07.466 ******* 2019-11-25T09:09:36.116 INFO:teuthology.task.ansible.out: TASK [common : Get the current timezone] *************************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.153 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.067) 0:02:07.534 ******* 2019-11-25T09:09:36.180 INFO:teuthology.task.ansible.out: TASK [common : Set the timezone] *********************************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.219 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.065) 0:02:07.599 ******* 2019-11-25T09:09:36.247 INFO:teuthology.task.ansible.out: TASK [common : Add base OpenSUSE Leap repo] ************************************ skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.284 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.066) 0:02:07.665 ******* 2019-11-25T09:09:36.312 INFO:teuthology.task.ansible.out: TASK [common : Add updates OpenSUSE Leap repo] ********************************* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.352 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.065) 0:02:07.730 ******* 2019-11-25T09:09:36.376 INFO:teuthology.task.ansible.out: TASK [common : Refresh repos] ************************************************** 2019-11-25T09:09:36.379 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [common : Install Kerberos Packages (RedHat)] ***************************** ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-11-25T09:09:36.417 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.067) 0:02:07.797 ******* 2019-11-25T09:09:36.452 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.034) 0:02:07.832 ******* 2019-11-25T09:09:36.457 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-11-25T09:09:36.459 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.495 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.043) 0:02:07.876 ******* 2019-11-25T09:09:36.532 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-11-25T09:09:36.534 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.569 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.073) 0:02:07.950 ******* 2019-11-25T09:09:36.595 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-11-25T09:09:36.597 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.632 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.063) 0:02:08.013 ******* 2019-11-25T09:09:36.659 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-11-25T09:09:36.660 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:36.698 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:36 +0000 (0:00:00.066) 0:02:08.079 ******* 2019-11-25T09:09:37.529 INFO:teuthology.task.ansible.out: TASK [common : Set the hardware clock] ***************************************** 2019-11-25T09:09:37.531 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": ["hwclock", "--systohc"], "delta": "0:00:00.784711", "end": "2019-11-25 09:09:37.500493", "rc": 0, "start": "2019-11-25 09:09:36.715782", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:09:37.577 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:37 +0000 (0:00:00.878) 0:02:08.958 ******* 2019-11-25T09:09:37.586 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-11-25T09:09:37.587 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-11-25T09:09:37.629 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:37 +0000 (0:00:00.052) 0:02:09.010 ******* 2019-11-25T09:09:38.063 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (RedHat)] ***************************** 2019-11-25T09:09:38.065 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["krb5-workstation-1.15.1-37.el7_7.2.x86_64 providing krb5-workstation is already installed"]} 2019-11-25T09:09:38.100 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:38 +0000 (0:00:00.471) 0:02:09.481 ******* 2019-11-25T09:09:38.126 INFO:teuthology.task.ansible.out: TASK [common : Clean apt cache] ************************************************ 2019-11-25T09:09:38.128 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:38.161 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:38 +0000 (0:00:00.061) 0:02:09.542 ******* 2019-11-25T09:09:38.186 INFO:teuthology.task.ansible.out: TASK [common : Update apt cache] *********************************************** 2019-11-25T09:09:38.188 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:38.222 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:38 +0000 (0:00:00.060) 0:02:09.603 ******* 2019-11-25T09:09:38.258 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (Debian)] ***************************** 2019-11-25T09:09:38.259 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:38.293 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:38 +0000 (0:00:00.071) 0:02:09.674 ******* 2019-11-25T09:09:38.319 INFO:teuthology.task.ansible.out: TASK [common : Install Kerberos Packages (OpenSUSE Leap)] ********************** 2019-11-25T09:09:38.321 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:38.355 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:38 +0000 (0:00:00.061) 0:02:09.736 ******* 2019-11-25T09:09:39.282 INFO:teuthology.task.ansible.out: TASK [common : Add krb5 config file] ******************************************* 2019-11-25T09:09:39.283 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "b3c1a33d19a4e448c29374429629dff57657b576", "dest": "/etc/krb5.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/krb5.conf", "secontext": "system_u:object_r:krb5_conf_t:s0", "size": 131, "state": "file", "uid": 0} 2019-11-25T09:09:39.321 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:39 +0000 (0:00:00.965) 0:02:10.702 ******* 2019-11-25T09:09:42.765 INFO:teuthology.task.ansible.out: TASK [common : Make sure smartmontools is installed] *************************** 2019-11-25T09:09:42.765 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-11-25T09:09:42.813 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:42 +0000 (0:00:03.491) 0:02:14.193 ******* 2019-11-25T09:09:43.232 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing smartmontools are up to date", ""]} 2019-11-25T09:09:43.277 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:43 +0000 (0:00:00.463) 0:02:14.657 ******* 2019-11-25T09:09:43.799 INFO:teuthology.task.ansible.out: TASK [common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/.] *** ok: [smithi081.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-11-25T09:09:44.223 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=megacli) => {"ansible_loop_var": "item", "changed": false, "checksum": "cb3ef2aa6a3e3002b234f166e739a39b1f3a1bde", "dest": "/usr/sbin/megacli", "gid": 0, "group": "root", "item": "megacli", "mode": "0755", "owner": "root", "path": "/usr/sbin/megacli", "secontext": "system_u:object_r:bin_t:s0", "size": 2716736, "state": "file", "uid": 0} 2019-11-25T09:09:44.970 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-11-25T09:09:45.322 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=cli64) => {"ansible_loop_var": "item", "changed": false, "checksum": "a3b8773b14e8594ffbd9fc156c09436eb0184bc7", "dest": "/usr/sbin/cli64", "gid": 0, "group": "root", "item": "cli64", "mode": "0755", "owner": "root", "path": "/usr/sbin/cli64", "secontext": "system_u:object_r:bin_t:s0", "size": 1952038, "state": "file", "uid": 0} 2019-11-25T09:09:45.860 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-11-25T09:09:45.909 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:45 +0000 (0:00:02.632) 0:02:17.290 ******* 2019-11-25T09:09:46.185 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=nvme) => {"ansible_loop_var": "item", "changed": false, "checksum": "98735dcacd6fec55dd92b67281b478603b7955b1", "dest": "/usr/sbin/nvme", "gid": 0, "group": "root", "item": "nvme", "mode": "0755", "owner": "root", "path": "/usr/sbin/nvme", "secontext": "system_u:object_r:bin_t:s0", "size": 580432, "state": "file", "uid": 0} 2019-11-25T09:09:46.242 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:46 +0000 (0:00:00.332) 0:02:17.623 ******* 2019-11-25T09:09:46.259 INFO:teuthology.task.ansible.out: TASK [common : Create /usr/libexec.] ******************************************* 2019-11-25T09:09:46.264 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-25T09:09:46.298 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:46 +0000 (0:00:00.056) 0:02:17.679 ******* 2019-11-25T09:09:46.568 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/libexec", "secontext": "system_u:object_r:bin_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-25T09:09:46.620 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:46 +0000 (0:00:00.309) 0:02:17.989 ******* 2019-11-25T09:09:47.211 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-11-25T09:09:47.212 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-11-25T09:09:47.463 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=smart.sh) => {"ansible_loop_var": "item", "changed": false, "checksum": "9a447e184389aadb9aaf1e77f30f0a7f8b716c4e", "dest": "/usr/libexec/smart.sh", "gid": 0, "group": "root", "item": "smart.sh", "mode": "0755", "owner": "root", "path": "/usr/libexec/smart.sh", "secontext": "system_u:object_r:bin_t:s0", "size": 12725, "state": "file", "uid": 0} 2019-11-25T09:09:48.092 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-11-25T09:09:48.284 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=raid.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9d9a89805d8a9e4c7281934e887da31f7ac464fb", "dest": "/usr/libexec/raid.pl", "gid": 0, "group": "root", "item": "raid.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/raid.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 8602, "state": "file", "uid": 0} 2019-11-25T09:09:48.939 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-11-25T09:09:48.981 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:48 +0000 (0:00:02.372) 0:02:20.361 ******* 2019-11-25T09:09:49.007 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-11-25T09:09:49.008 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.052 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.071) 0:02:20.433 ******* 2019-11-25T09:09:49.105 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** 2019-11-25T09:09:49.110 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-11-25T09:09:49.145 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.092) 0:02:20.526 ******* 2019-11-25T09:09:49.158 INFO:teuthology.task.ansible.out: TASK [common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/.] *** 2019-11-25T09:09:49.159 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=diskusage.pl) => {"ansible_loop_var": "item", "changed": false, "checksum": "9c74425885a83f23bc37817de46979ca5173ccb7", "dest": "/usr/libexec/diskusage.pl", "gid": 0, "group": "root", "item": "diskusage.pl", "mode": "0755", "owner": "root", "path": "/usr/libexec/diskusage.pl", "secontext": "system_u:object_r:bin_t:s0", "size": 3605, "state": "file", "uid": 0} 2019-11-25T09:09:49.199 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.053) 0:02:20.580 ******* 2019-11-25T09:09:49.228 INFO:teuthology.task.ansible.out: TASK [common : Include apt_system vars] **************************************** 2019-11-25T09:09:49.230 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.267 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.067) 0:02:20.647 ******* 2019-11-25T09:09:49.305 INFO:teuthology.task.ansible.out: TASK [common : Include yum_systems vars] *************************************** ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"nagios_plugins_directory": "/usr/lib64/nagios/plugins", "nrpe_group": "nrpe", "nrpe_packages": ["nagios-common", "nrpe", "nagios-plugins", "nagios-plugins-load"], "nrpe_service_name": "nrpe", "nrpe_user": "nrpe"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/common/vars/yum_systems.yml"], "changed": false} 2019-11-25T09:09:49.341 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.074) 0:02:20.722 ******* 2019-11-25T09:09:49.474 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-11-25T09:09:49.476 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-11-25T09:09:49.513 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.171) 0:02:20.893 ******* 2019-11-25T09:09:49.561 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-11-25T09:09:49.563 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.573 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.585 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.596 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.635 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.122) 0:02:21.016 ******* 2019-11-25T09:09:49.653 INFO:teuthology.task.ansible.out: TASK [common : Check for epel] ************************************************* 2019-11-25T09:09:49.655 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} 2019-11-25T09:09:49.700 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.064) 0:02:21.080 ******* 2019-11-25T09:09:49.748 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe without epel] ************************************** 2019-11-25T09:09:49.750 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-common-4.0.8-2.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.760 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nrpe-2.15-7.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.771 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.782 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm) => {"ansible_loop_var": "item", "changed": false, "item": "http://apt-mirror.front.sepia.ceph.com/lab-extras/rhel7/x86_64/nagios-plugins-load-2.0.3-3.el7.x86_64.rpm", "skip_reason": "Conditional result was False"} 2019-11-25T09:09:49.826 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:49 +0000 (0:00:00.126) 0:02:21.207 ******* 2019-11-25T09:09:53.048 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-11-25T09:09:53.050 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-11-25T09:09:53.086 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:53 +0000 (0:00:03.259) 0:02:24.467 ******* 2019-11-25T09:09:53.112 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-11-25T09:09:53.114 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:53.149 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:53 +0000 (0:00:00.062) 0:02:24.530 ******* 2019-11-25T09:09:53.177 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (RHEL/CentOS)] ************ 2019-11-25T09:09:53.178 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing nagios-common are up to date", "All packages providing nrpe are up to date", "All packages providing nagios-plugins are up to date", "All packages providing nagios-plugins-load are up to date", ""]} 2019-11-25T09:09:53.227 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:53 +0000 (0:00:00.077) 0:02:24.608 ******* 2019-11-25T09:09:53.253 INFO:teuthology.task.ansible.out: TASK [common : Install nrpe package and dependencies (Ubuntu)] ***************** 2019-11-25T09:09:53.255 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:53.293 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:53 +0000 (0:00:00.066) 0:02:24.674 ******* 2019-11-25T09:09:54.027 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios sudoers.d for raid utilities.] ******************** 2019-11-25T09:09:54.028 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-11-25T09:09:54.065 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:54 +0000 (0:00:00.772) 0:02:25.446 ******* 2019-11-25T09:09:54.155 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "b423b985ccbb69b65a711d372645a5f704b30e8b", "dest": "/etc/sudoers.d/90-nagios", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers.d/90-nagios", "secontext": "system_u:object_r:etc_t:s0", "size": 178, "state": "file", "uid": 0} 2019-11-25T09:09:54.195 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:54 +0000 (0:00:00.129) 0:02:25.576 ******* 2019-11-25T09:09:54.903 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-11-25T09:09:54.903 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-11-25T09:09:54.942 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:54 +0000 (0:00:00.747) 0:02:26.323 ******* 2019-11-25T09:09:54.969 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ 2019-11-25T09:09:54.971 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:55.007 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:55 +0000 (0:00:00.064) 0:02:26.388 ******* 2019-11-25T09:09:55.067 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios check_mem script] ********************************* 2019-11-25T09:09:55.068 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "39df629b10ded370443e2e4c84d690332c95104d", "dest": "/usr/lib64/nagios/plugins/check_mem.sh", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib64/nagios/plugins/check_mem.sh", "secontext": "system_u:object_r:nagios_unconfined_plugin_exec_t:s0", "size": 1964, "state": "file", "uid": 0} 2019-11-25T09:09:55.107 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:55 +0000 (0:00:00.099) 0:02:26.488 ******* 2019-11-25T09:09:55.134 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (Ubuntu)] ************************ skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:55.171 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:55 +0000 (0:00:00.063) 0:02:26.552 ******* 2019-11-25T09:09:55.337 INFO:teuthology.task.ansible.out: TASK [common : Configure nagios nrpe settings (RHEL/CentOS)] ******************* 2019-11-25T09:09:55.339 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:09:55.390 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:55 +0000 (0:00:00.213) 0:02:26.766 ******* 2019-11-25T09:09:55.494 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:09:55.537 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:55 +0000 (0:00:00.152) 0:02:26.918 ******* 2019-11-25T09:09:55.750 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** fatal: [smithi153.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-11-25T09:09:55.803 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:55 +0000 (0:00:00.265) 0:02:27.184 ******* 2019-11-25T09:09:55.841 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** 2019-11-25T09:09:55.842 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:55.897 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:55 +0000 (0:00:00.093) 0:02:27.277 ******* 2019-11-25T09:09:55.903 INFO:teuthology.task.ansible.out: TASK [common : Check firewalld status] ***************************************** 2019-11-25T09:09:55.904 INFO:teuthology.task.ansible.out:fatal: [smithi081.front.sepia.ceph.com]: FAILED! => {"censored": "the output has been hidden due to the fact that 'no_log: true' was specified for this result", "changed": true} ...ignoring 2019-11-25T09:09:55.953 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:55 +0000 (0:00:00.056) 0:02:27.334 ******* 2019-11-25T09:09:56.000 INFO:teuthology.task.ansible.out: TASK [common : Open nrpe port if firewalld enabled] **************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:09:56.029 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:56 +0000 (0:00:00.076) 0:02:27.410 ******* 2019-11-25T09:09:56.775 INFO:teuthology.task.ansible.out: TASK [common : Upload nagios nrpe config.] ************************************* 2019-11-25T09:09:56.776 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-11-25T09:09:56.820 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:56 +0000 (0:00:00.791) 0:02:28.201 ******* 2019-11-25T09:09:56.941 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "333141ba5e85aea94f51511a52d9b115d2e0112f", "dest": "/etc/nagios/nrpe.cfg", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/nagios/nrpe.cfg", "secontext": "system_u:object_r:nrpe_etc_t:s0", "size": 1174, "state": "file", "uid": 0} 2019-11-25T09:09:56.981 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:56 +0000 (0:00:00.160) 0:02:28.362 ******* 2019-11-25T09:09:57.648 INFO:teuthology.task.ansible.out: TASK [common : Make sure nagios nrpe service is running.] ********************** 2019-11-25T09:09:57.650 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-11-25 09:07:01 UTC", "ActiveEnterTimestampMonotonic": "19231620", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "nss-lookup.target basic.target network.target var-run.mount system.slice time-sync.target local-fs.target systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-11-25 09:07:01 UTC", "AssertTimestampMonotonic": "19230967", "Before": "shutdown.target multi-user.target getty@tty1.service xdm.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-11-25 09:07:01 UTC", "ConditionTimestampMonotonic": "19230965", "Conflicts": "nrpe.socket shutdown.target", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8970", "ExecMainStartTimestamp": "Mon 2019-11-25 09:07:01 UTC", "ExecMainStartTimestampMonotonic": "19231562", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-11-25T09:09:57.652 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Mon 2019-11-25 09:07:01 UTC] ; stop_time=[n/a] ; pid=8970 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-11-25 09:07:01 UTC", "InactiveExitTimestampMonotonic": "19231620", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8970", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-11-25T09:09:57.653 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-11-25 09:07:01 UTC", "WatchdogTimestampMonotonic": "19231601", "WatchdogUSec": "0"}} 2019-11-25T09:09:57.690 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:57 +0000 (0:00:00.708) 0:02:29.071 ******* 2019-11-25T09:09:57.699 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "enabled": true, "name": "nrpe", "state": "started", "status": {"ActiveEnterTimestamp": "Mon 2019-11-25 09:07:08 UTC", "ActiveEnterTimestampMonotonic": "19611599", "ActiveExitTimestampMonotonic": "0", "ActiveState": "active", "After": "basic.target network.target nss-lookup.target time-sync.target systemd-journald.socket var-run.mount system.slice local-fs.target", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "yes", "AssertTimestamp": "Mon 2019-11-25 09:07:08 UTC", "AssertTimestampMonotonic": "19611034", "Before": "getty@tty1.service multi-user.target xdm.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "yes", "ConditionTimestamp": "Mon 2019-11-25 09:07:08 UTC", "ConditionTimestampMonotonic": "19611033", "Conflicts": "shutdown.target nrpe.socket", "ControlGroup": "/system.slice/nrpe.service", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Nagios Remote Program Executor", "DevicePolicy": "auto", "Documentation": "http://www.nagios.org/documentation", "EnvironmentFile": "/etc/sysconfig/nrpe (ignore_errors=no)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "8959", "ExecMainStartTimestamp": "Mon 2019-11-25 09:07:08 UTC", "ExecMainStartTimestampMonotonic": "19611551", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/nrpe ; argv[]=/usr/sbin/nrpe -c /etc/nagios/nrpe.cfg -f $NRPE_SSL_OPT ; ignore_err 2019-11-25T09:09:57.700 INFO:teuthology.task.ansible.out:ors=no ; start_time=[Mon 2019-11-25 09:07:08 UTC] ; stop_time=[n/a] ; pid=8959 ; code=(null) ; status=0/0 }", "ExecStopPost": "{ path=/bin/rm ; argv[]=/bin/rm -f /var/run/nrpe/nrpe.pid ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/nrpe.service", "Group": "nrpe", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "nrpe.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestamp": "Mon 2019-11-25 09:07:08 UTC", "InactiveExitTimestampMonotonic": "19611599", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "8959", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "nrpe.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "network-online.target basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOn 2019-11-25T09:09:57.702 INFO:teuthology.task.ansible.out:ly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "running", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "simple", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "User": "nrpe", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestamp": "Mon 2019-11-25 09:07:08 UTC", "WatchdogTimestampMonotonic": "19611582", "WatchdogUSec": "0"}} 2019-11-25T09:09:57.738 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:57 +0000 (0:00:00.047) 0:02:29.119 ******* 2019-11-25T09:09:58.024 INFO:teuthology.task.ansible.out: TASK [common : Get SELinux status] ********************************************* 2019-11-25T09:09:58.025 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.004795", "end": "2019-11-25 09:09:57.991122", "rc": 0, "start": "2019-11-25 09:09:57.986327", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-11-25T09:09:58.066 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:58 +0000 (0:00:00.327) 0:02:29.447 ******* 2019-11-25T09:09:58.074 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": ["getenforce"], "delta": "0:00:00.003764", "end": "2019-11-25 09:09:58.024481", "rc": 0, "start": "2019-11-25 09:09:58.020717", "stderr": "", "stderr_lines": [], "stdout": "Permissive", "stdout_lines": ["Permissive"]} 2019-11-25T09:09:58.124 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:58 +0000 (0:00:00.057) 0:02:29.504 ******* 2019-11-25T09:09:58.547 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Install SELinux dependencies] **************************** 2019-11-25T09:09:58.547 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-11-25T09:09:58.589 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:58 +0000 (0:00:00.465) 0:02:29.970 ******* 2019-11-25T09:09:58.637 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["libsemanage-python-2.5-14.el7.x86_64 providing libsemanage-python is already installed", "policycoreutils-python-2.5-29.el7.x86_64 providing policycoreutils-python is already installed"]} 2019-11-25T09:09:58.677 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:09:58 +0000 (0:00:00.088) 0:02:30.058 ******* 2019-11-25T09:10:01.025 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Ensure SELinux policy is up to date] ********************* 2019-11-25T09:10:01.026 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing selinux-policy-targeted are up to date", ""]} 2019-11-25T09:10:01.064 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:01 +0000 (0:00:02.387) 0:02:32.445 ******* 2019-11-25T09:10:01.095 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": [], "updated": []}, "msg": "", "rc": 0, "results": ["All packages providing selinux-policy-targeted are up to date", ""]} 2019-11-25T09:10:01.141 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:01 +0000 (0:00:00.076) 0:02:32.522 ******* 2019-11-25T09:10:02.425 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Set SELinux boolean nagios_run_sudo true] **************** 2019-11-25T09:10:02.427 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-11-25T09:10:02.461 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:02 +0000 (0:00:01.319) 0:02:33.842 ******* 2019-11-25T09:10:02.466 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "name": "nagios_run_sudo", "persistent": true, "state": true} 2019-11-25T09:10:02.506 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:02 +0000 (0:00:00.044) 0:02:33.887 ******* 2019-11-25T09:10:10.779 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove SELinux policy package] *************************** 2019-11-25T09:10:10.781 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:07.976639", "end": "2019-11-25 09:10:10.738235", "failed_when_result": false, "rc": 0, "start": "2019-11-25 09:10:02.761596", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-11-25T09:10:10.823 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:10 +0000 (0:00:08.317) 0:02:42.204 ******* 2019-11-25T09:10:10.861 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-r", "nrpe"], "delta": "0:00:08.003459", "end": "2019-11-25 09:10:10.818498", "failed_when_result": false, "rc": 0, "start": "2019-11-25 09:10:02.815039", "stderr": "", "stderr_lines": [], "stdout": "libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority).", "stdout_lines": ["libsemanage.semanage_direct_remove_key: Removing last nrpe module (no other nrpe module exists at another priority)."]} 2019-11-25T09:10:10.903 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:10 +0000 (0:00:00.079) 0:02:42.283 ******* 2019-11-25T09:10:11.763 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Copy SELinux type enforcement file] ********************** ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-11-25T09:10:11.805 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:11 +0000 (0:00:00.902) 0:02:43.186 ******* 2019-11-25T09:10:11.875 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "99d9b2ea41fb2d6613898129718064833d2de428", "dest": "/tmp/nrpe.te", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/tmp/nrpe.te", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 934, "state": "file", "uid": 0} 2019-11-25T09:10:11.914 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:11 +0000 (0:00:00.109) 0:02:43.295 ******* 2019-11-25T09:10:12.148 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Compile SELinux module file] ***************************** 2019-11-25T09:10:12.148 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.019232", "end": "2019-11-25 09:10:12.120119", "rc": 0, "start": "2019-11-25 09:10:12.100887", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-11-25T09:10:12.202 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:12 +0000 (0:00:00.287) 0:02:43.583 ******* 2019-11-25T09:10:12.268 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": ["checkmodule", "-M", "-m", "-o", "/tmp/nrpe.mod", "/tmp/nrpe.te"], "delta": "0:00:00.016019", "end": "2019-11-25 09:10:12.236060", "rc": 0, "start": "2019-11-25 09:10:12.220041", "stderr": "", "stderr_lines": [], "stdout": "checkmodule: loading policy configuration from /tmp/nrpe.te\ncheckmodule: policy configuration loaded\ncheckmodule: writing binary representation (version 19) to /tmp/nrpe.mod", "stdout_lines": ["checkmodule: loading policy configuration from /tmp/nrpe.te", "checkmodule: policy configuration loaded", "checkmodule: writing binary representation (version 19) to /tmp/nrpe.mod"]} 2019-11-25T09:10:12.306 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:12 +0000 (0:00:00.103) 0:02:43.687 ******* 2019-11-25T09:10:12.538 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Build SELinux policy package] **************************** 2019-11-25T09:10:12.544 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.013868", "end": "2019-11-25 09:10:12.506094", "rc": 0, "start": "2019-11-25 09:10:12.492226", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:10:12.614 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:12 +0000 (0:00:00.308) 0:02:43.995 ******* 2019-11-25T09:10:12.655 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule_package", "-o", "/tmp/nrpe.pp", "-m", "/tmp/nrpe.mod"], "delta": "0:00:00.011402", "end": "2019-11-25 09:10:12.616826", "rc": 0, "start": "2019-11-25 09:10:12.605424", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:10:12.709 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:12 +0000 (0:00:00.094) 0:02:44.090 ******* 2019-11-25T09:10:20.849 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Load SELinux policy package] ***************************** changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.823166", "end": "2019-11-25 09:10:20.810155", "rc": 0, "start": "2019-11-25 09:10:12.986989", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:10:20.850 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": ["semodule", "-i", "/tmp/nrpe.pp"], "delta": "0:00:07.893696", "end": "2019-11-25 09:10:20.818248", "rc": 0, "start": "2019-11-25 09:10:12.924552", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:10:20.889 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:20 +0000 (0:00:08.180) 0:02:52.270 ******* 2019-11-25T09:10:20.890 INFO:teuthology.task.ansible.out: 2019-11-25T09:10:20.959 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:20 +0000 (0:00:00.066) 0:02:52.337 ******* 2019-11-25T09:10:21.236 INFO:teuthology.task.ansible.out: TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-11-25T09:10:21.282 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.322) 0:02:52.659 ******* 2019-11-25T09:10:21.306 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-11-25T09:10:21.356 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.070) 0:02:52.729 ******* TASK [common : nrpe - Remove temporary files] ********************************** ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "path": "/tmp/nrpe.*", "state": "absent"} 2019-11-25T09:10:21.403 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.051) 0:02:52.780 ******* 2019-11-25T09:10:21.410 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* ok: [smithi081.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-11-25T09:10:21.451 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.051) 0:02:52.832 ******* 2019-11-25T09:10:21.459 INFO:teuthology.task.ansible.out: TASK [testnode : Include package type specific vars.] ************************** ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"ceph_dependency_packages_to_remove": ["boost-random", "boost-program-options", "leveldb", "xmlstarlet", "python-jinja2", "python-ceph", "python-flask", "python-requests", "boost-random", "python-urllib3", "python-babel", "hdparm", "python-markupsafe", "python-werkzeug", "python-itsdangerous"], "ceph_packages_to_remove": ["ceph", "ceph-base", "ceph-selinux", "ceph-common", "ceph-debuginfo", "ceph-release", "libcephfs1", "ceph-radosgw", "python-ceph", "python-rados", "python-rbd", "python-cephfs", "librbd1", "librados2", "mod_fastcgi", "iozone"], "ntp_service_name": "ntpd", "packages_to_remove": ["device-mapper-multipath"], "pip_packages_to_install": ["remoto>=0.0.35"], "ssh_service_name": "sshd"}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/yum_systems.yml"], "changed": false} 2019-11-25T09:10:21.497 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.046) 0:02:52.878 ******* 2019-11-25T09:10:21.513 INFO:teuthology.task.ansible.out: TASK [testnode : Including major version specific variables.] ****************** 2019-11-25T09:10:21.514 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-11-25T09:10:21.555 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.057) 0:02:52.936 ******* 2019-11-25T09:10:21.561 INFO:teuthology.task.ansible.out: TASK [testnode : Including distro specific variables.] ************************* 2019-11-25T09:10:21.562 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-11-25T09:10:21.604 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.048) 0:02:52.985 ******* 2019-11-25T09:10:21.612 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ ok: [smithi081.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-11-25T09:10:21.655 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.045) 0:02:53.030 ******* TASK [testnode : Including major version specific variables.] ****************** 2019-11-25T09:10:21.656 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml) => {"ansible_facts": {"epel_packages": ["cryptopp-devel", "cryptopp", "fcgi", "dbench", "fuse-sshfs", "bonnie++", "perl-JSON-XS"], "packages": ["@core", "@base", "yum-plugin-priorities", "yum-plugin-fastestmirror", "redhat-lsb", "sysstat", "gdb", "git-all", "python-configobj", "gcc-c++", "libedit", "openssl098e", "boost-thread", "xfsprogs", "gdisk", "parted", "libgcrypt", "fuse", "fuse-libs", "python-virtualenv", "openssl", "libuuid", "btrfs-progs", "attr", "valgrind", "python-nose", "mpich", "podman", "ant", "iozone", "libtool", "automake", "gettext", "libuuid-devel", "libacl-devel", "bc", "xfsdump", "xfsprogs-devel", "blktrace", "numpy", "python-matplotlib", "qemu-kvm", "usbredir", "genisoimage", "httpd", "httpd-devel", "httpd-tools", "mod_ssl", "mod_fastcgi-2.4.7-1.ceph.el7.centos", "libev-devel", "perl-XML-Twig", "java-1.6.0-openjdk-devel", "junit4", "nfs-utils", "ncurses-devel", "python-devel", "python-virtualenv", "perl-CPAN"], "yum_repos": {"centos7-fcgi-ceph": {"baseurl": "http://{{ gitbuilder_host }}/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}, "lab-extras": {"baseurl": "http://{{ mirror_host }}/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/centos_7.yml"} 2019-11-25T09:10:21.694 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.045) 0:02:53.075 ******* 2019-11-25T09:10:21.731 INFO:teuthology.task.ansible.out: TASK [testnode : Including version specific variables.] ************************ 2019-11-25T09:10:21.733 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml) => {"ansible_facts": {}, "ansible_included_var_files": ["/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"], "ansible_loop_var": "item", "changed": false, "item": "/home/teuthworker/src/git.ceph.com_git_ceph-cm-ansible_master/roles/testnode/vars/empty.yml"} 2019-11-25T09:10:21.784 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:21 +0000 (0:00:00.087) 0:02:53.162 ******* 2019-11-25T09:10:21.967 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the sudo group exists.] ******************************** 2019-11-25T09:10:21.968 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-11-25T09:10:22.007 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.225) 0:02:53.388 ******* 2019-11-25T09:10:22.086 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "gid": 1000, "name": "sudo", "state": "present", "system": false} 2019-11-25T09:10:22.135 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.127) 0:02:53.516 ******* 2019-11-25T09:10:22.327 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-11-25T09:10:22.329 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-11-25T09:10:22.378 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.230) 0:02:53.747 ******* 2019-11-25T09:10:22.394 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-11-25T09:10:22.395 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:22.436 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.070) 0:02:53.817 ******* 2019-11-25T09:10:22.442 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the teuthology_user group exists.] ********************* 2019-11-25T09:10:22.442 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "name": "ubuntu", "state": "present", "system": false} 2019-11-25T09:10:22.482 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.045) 0:02:53.862 ******* 2019-11-25T09:10:22.487 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-11-25T09:10:22.487 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:22.525 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.043) 0:02:53.905 ******* 2019-11-25T09:10:22.530 INFO:teuthology.task.ansible.out: TASK [testnode : Create the teuthology user.] ********************************** 2019-11-25T09:10:22.532 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:22.569 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.043) 0:02:53.949 ******* 2019-11-25T09:10:22.590 INFO:teuthology.task.ansible.out: TASK [testnode : Delete the teuthology users password.] ************************ 2019-11-25T09:10:22.592 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:22.627 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.059) 0:02:54.008 ******* 2019-11-25T09:10:22.891 INFO:teuthology.task.ansible.out: TASK [testnode : Add a user for xfstests to test user quotas.] ***************** 2019-11-25T09:10:22.892 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-11-25T09:10:22.929 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:22 +0000 (0:00:00.301) 0:02:54.309 ******* 2019-11-25T09:10:22.998 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"append": false, "changed": false, "comment": "", "group": 10101, "home": "/home/fsgqa", "move_home": false, "name": "fsgqa", "shell": "/bin/bash", "state": "present", "uid": 10101} 2019-11-25T09:10:23.036 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:23 +0000 (0:00:00.107) 0:02:54.417 ******* 2019-11-25T09:10:23.949 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-11-25T09:10:23.996 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:23 +0000 (0:00:00.960) 0:02:55.377 ******* 2019-11-25T09:10:24.026 INFO:teuthology.task.ansible.out: TASK [testnode : Set hostname var] ********************************************* 2019-11-25T09:10:24.027 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi081"}, "changed": false} 2019-11-25T09:10:24.081 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:24 +0000 (0:00:00.084) 0:02:55.462 ******* 2019-11-25T09:10:24.083 INFO:teuthology.task.ansible.out: 2019-11-25T09:10:24.089 INFO:teuthology.task.ansible.out: TASK [testnode : Set a high max open files limit for the teuthology user.] ***** 2019-11-25T09:10:24.089 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "dc140f4bb26594b8d5e0753bab8d93c0cced7772", "dest": "/etc/security/limits.d/ubuntu.conf", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/security/limits.d/ubuntu.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 112, "state": "file", "uid": 0} 2019-11-25T09:10:24.129 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:24 +0000 (0:00:00.047) 0:02:55.510 ******* 2019-11-25T09:10:24.172 INFO:teuthology.task.ansible.out: 2019-11-25T09:10:24.179 INFO:teuthology.task.ansible.out:TASK [testnode : Set hostname var] ********************************************* 2019-11-25T09:10:24.180 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"hostname": "smithi153"}, "changed": false} 2019-11-25T09:10:24.217 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:24 +0000 (0:00:00.088) 0:02:55.598 ******* 2019-11-25T09:10:25.016 INFO:teuthology.task.ansible.out: TASK [testnode : Set the system's hostname] ************************************ changed: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi081.front.sepia.ceph.com", "ansible_hostname": "smithi081", "ansible_nodename": "smithi081"}, "changed": true, "name": "smithi081"} 2019-11-25T09:10:25.055 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:25 +0000 (0:00:00.838) 0:02:56.436 ******* 2019-11-25T09:10:25.063 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"ansible_domain": "front.sepia.ceph.com", "ansible_fqdn": "smithi153.front.sepia.ceph.com", "ansible_hostname": "smithi153", "ansible_nodename": "smithi153"}, "changed": true, "name": "smithi153"} 2019-11-25T09:10:25.112 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:25 +0000 (0:00:00.056) 0:02:56.493 ******* 2019-11-25T09:10:25.898 INFO:teuthology.task.ansible.out: TASK [testnode : Upload distro major version specific sshd_config] ************* 2019-11-25T09:10:25.898 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-11-25T09:10:25.940 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:25 +0000 (0:00:00.827) 0:02:57.321 ******* 2019-11-25T09:10:25.989 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "28e3f1fe12efa510f1199f3ab5f2a5a0feb586ff", "dest": "/etc/ssh/sshd_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/sshd_config", "secontext": "system_u:object_r:etc_t:s0", "size": 1042, "state": "file", "uid": 0} 2019-11-25T09:10:26.028 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:26 +0000 (0:00:00.088) 0:02:57.409 ******* 2019-11-25T09:10:26.830 INFO:teuthology.task.ansible.out: TASK [testnode : Upload ssh_config] ******************************************** 2019-11-25T09:10:26.830 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-11-25T09:10:26.867 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:26 +0000 (0:00:00.838) 0:02:58.248 ******* 2019-11-25T09:10:26.967 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "58d78b02605c8368f1913de12e145f4ad930fafc", "dest": "/etc/ssh/ssh_config", "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/etc/ssh/ssh_config", "secontext": "system_u:object_r:etc_t:s0", "size": 570, "state": "file", "uid": 0} 2019-11-25T09:10:27.004 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:26 +0000 (0:00:00.136) 0:02:58.384 ******* 2019-11-25T09:10:27.790 INFO:teuthology.task.ansible.out: TASK [testnode : Add ssh pubkeys] ********************************************** 2019-11-25T09:10:27.791 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-11-25T09:10:27.831 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:27 +0000 (0:00:00.827) 0:02:59.212 ******* 2019-11-25T09:10:27.944 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "comment": null, "exclusive": false, "follow": false, "key": "https://raw.githubusercontent.com/ceph/keys/autogenerated/ssh/@all.pub", "key_options": null, "keyfile": "/home/ubuntu/.ssh/authorized_keys", "manage_dir": true, "path": null, "state": "present", "user": "ubuntu", "validate_certs": true} 2019-11-25T09:10:27.993 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:27 +0000 (0:00:00.161) 0:02:59.374 ******* 2019-11-25T09:10:28.723 INFO:teuthology.task.ansible.out: TASK [testnode : Create remote.conf] ******************************************* 2019-11-25T09:10:28.724 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-11-25T09:10:28.769 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:28 +0000 (0:00:00.775) 0:03:00.149 ******* 2019-11-25T09:10:28.900 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "68d0b8ee2f88a77895f257aa69f32a42e47f455b", "dest": "/etc/security/limits.d/remote.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.d/remote.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 116, "state": "file", "uid": 0} 2019-11-25T09:10:28.937 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:28 +0000 (0:00:00.168) 0:03:00.318 ******* 2019-11-25T09:10:29.083 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-11-25T09:10:29.084 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-11-25T09:10:29.122 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:29 +0000 (0:00:00.184) 0:03:00.503 ******* 2019-11-25T09:10:29.274 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "dest": "/etc/fuse.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_t:s0", "size": 0, "state": "file", "uid": 0} 2019-11-25T09:10:29.311 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:29 +0000 (0:00:00.189) 0:03:00.692 ******* 2019-11-25T09:10:29.432 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* 2019-11-25T09:10:29.434 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-11-25T09:10:29.473 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:29 +0000 (0:00:00.158) 0:03:00.850 ******* 2019-11-25T09:10:29.662 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "gid": 10103, "name": "kvm", "state": "present", "system": false} 2019-11-25T09:10:29.701 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:29 +0000 (0:00:00.230) 0:03:01.081 ******* 2019-11-25T09:10:29.814 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* ok: [smithi081.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-11-25T09:10:29.848 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:29 +0000 (0:00:00.148) 0:03:01.229 ******* 2019-11-25T09:10:30.057 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"append": true, "changed": false, "comment": "", "group": 1215, "groups": "kvm,disk", "home": "/home/ubuntu", "move_home": false, "name": "ubuntu", "shell": "/bin/bash", "state": "present", "uid": 1000} 2019-11-25T09:10:30.095 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:30 +0000 (0:00:00.247) 0:03:01.476 ******* 2019-11-25T09:10:30.789 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-11-25T09:10:30.790 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-11-25T09:10:30.834 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:30 +0000 (0:00:00.738) 0:03:02.215 ******* 2019-11-25T09:10:31.070 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "be3223c8b0917bde02657a5f9ddd95277cc3f812", "dest": "/etc/sudoers", "gid": 0, "group": "root", "mode": "0440", "owner": "root", "path": "/etc/sudoers", "secontext": "system_u:object_r:etc_t:s0", "size": 1808, "state": "file", "uid": 0} 2019-11-25T09:10:31.108 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:31 +0000 (0:00:00.274) 0:03:02.489 ******* 2019-11-25T09:10:31.703 INFO:teuthology.task.ansible.out: 2019-11-25T09:10:31.705 INFO:teuthology.task.ansible.out:TASK [testnode : Configure /etc/security/limits.conf] ************************** ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-11-25T09:10:31.742 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:31 +0000 (0:00:00.633) 0:03:03.123 ******* 2019-11-25T09:10:32.060 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "12a2e17405b9e00ea2061e25929905d08e5fe19f", "dest": "/etc/security/limits.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/security/limits.conf", "secontext": "system_u:object_r:etc_t:s0", "size": 854, "state": "file", "uid": 0} 2019-11-25T09:10:32.097 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:32 +0000 (0:00:00.354) 0:03:03.478 ******* 2019-11-25T09:10:34.201 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-11-25T09:10:34.203 INFO:teuthology.task.ansible.out: [WARNING]: Consider using the yum, dnf or zypper module rather than running 'rpm'. If you need to use command because yum, dnf or zypper is insufficient you can add 'warn: false' to this command task or set 'command_warnings=False' in ansible.cfg to get rid of this message. changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.129002", "end": "2019-11-25 09:10:34.165489", "rc": 0, "start": "2019-11-25 09:10:32.036487", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:10:34.242 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:34 +0000 (0:00:02.144) 0:03:05.622 ******* 2019-11-25T09:10:34.268 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-11-25T09:10:34.322 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:34 +0000 (0:00:00.081) 0:03:05.703 ******* 2019-11-25T09:10:34.583 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-11-25T09:10:34.584 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": ["rpm", "--rebuilddb"], "delta": "0:00:02.153451", "end": "2019-11-25 09:10:34.549137", "rc": 0, "start": "2019-11-25 09:10:32.395686", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:10:34.647 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:34 +0000 (0:00:00.325) 0:03:06.028 ******* 2019-11-25T09:10:34.679 INFO:teuthology.task.ansible.out: TASK [testnode : Configure common additional repos in /etc/yum.repos.d/] ******* 2019-11-25T09:10:34.720 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:34 +0000 (0:00:00.072) 0:03:06.101 ******* 2019-11-25T09:10:35.225 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-11-25T09:10:35.226 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-11-25T09:10:35.630 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'CentOS 7 Local fastcgi Repo', u'baseurl': u'http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/'}, 'key': u'centos7-fcgi-ceph'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "58f6bfe213f63251ef8c4f03709a8fcef9b9e7bf", "dest": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "gid": 0, "group": "root", "item": {"key": "centos7-fcgi-ceph", "value": {"baseurl": "http://gitbuilder.ceph.com/mod_fastcgi-rpm-centos7-x86_64-basic/ref/master/", "enabled": 1, "gpgcheck": 0, "name": "CentOS 7 Local fastcgi Repo"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/centos7-fcgi-ceph.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 250, "state": "file", "uid": 0} 2019-11-25T09:10:36.084 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-11-25T09:10:36.126 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:01.406) 0:03:07.507 ******* 2019-11-25T09:10:36.164 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-11-25T09:10:36.166 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:36.209 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.082) 0:03:07.590 ******* 2019-11-25T09:10:36.251 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* skipping: [smithi081.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-11-25T09:10:36.261 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-11-25T09:10:36.302 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.093) 0:03:07.683 ******* 2019-11-25T09:10:36.330 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-11-25T09:10:36.332 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:36.367 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.064) 0:03:07.748 ******* 2019-11-25T09:10:36.405 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-11-25T09:10:36.440 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.073) 0:03:07.821 ******* 2019-11-25T09:10:36.539 INFO:teuthology.task.ansible.out: TASK [testnode : Configure version specific repos in /etc/yum.repos.d/] ******** 2019-11-25T09:10:36.539 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={'value': {u'gpgcheck': 0, u'enabled': 1, u'name': u'lab-extras', u'baseurl': u'http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/'}, 'key': u'lab-extras'}) => {"ansible_loop_var": "item", "changed": false, "checksum": "765bacbb40da31f952344b310a4be4322cbee8a4", "dest": "/etc/yum.repos.d/lab-extras.repo", "gid": 0, "group": "root", "item": {"key": "lab-extras", "value": {"baseurl": "http://apt-mirror.front.sepia.ceph.com/lab-extras/centos7/", "enabled": 1, "gpgcheck": 0, "name": "lab-extras"}}, "mode": "0644", "owner": "root", "path": "/etc/yum.repos.d/lab-extras.repo", "secontext": "system_u:object_r:system_conf_t:s0", "size": 209, "state": "file", "uid": 0} 2019-11-25T09:10:36.581 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.140) 0:03:07.962 ******* 2019-11-25T09:10:36.617 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG] ************************************************** 2019-11-25T09:10:36.619 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:36.658 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.077) 0:03:08.039 ******* 2019-11-25T09:10:36.695 INFO:teuthology.task.ansible.out: TASK [testnode : Install GPG keys] ********************************************* 2019-11-25T09:10:36.696 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-11-25T09:10:36.706 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=https://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "https://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-11-25T09:10:36.744 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.086) 0:03:08.125 ******* 2019-11-25T09:10:36.782 INFO:teuthology.task.ansible.out: TASK [testnode : Clean yum cache] ********************************************** 2019-11-25T09:10:36.784 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:36.833 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.088) 0:03:08.214 ******* 2019-11-25T09:10:36.869 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:10:36.871 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"skip_packaging": false}, "changed": false} 2019-11-25T09:10:36.909 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.076) 0:03:08.290 ******* 2019-11-25T09:10:36.915 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure yum_utils is present.] ********************************* 2019-11-25T09:10:36.916 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-11-25T09:10:36.959 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:36 +0000 (0:00:00.049) 0:03:08.340 ******* 2019-11-25T09:10:37.413 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["yum-utils-1.1.31-50.el7.noarch providing yum-utils is already installed"]} 2019-11-25T09:10:37.456 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:37 +0000 (0:00:00.496) 0:03:08.837 ******* 2019-11-25T09:10:37.598 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-11-25T09:10:37.598 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.323246", "end": "2019-11-25 09:10:37.562728", "rc": 0, "start": "2019-11-25 09:10:37.239482", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: packages.oit.ncsu.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.jaleco.com\n * updates: mirror.atlanticmetro.net\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: packages.oit.ncsu.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.jaleco.com", " * updates: mirror.atlanticmetro.net", "No unfinished transactions left."]} 2019-11-25T09:10:37.642 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:37 +0000 (0:00:00.186) 0:03:09.023 ******* 2019-11-25T09:10:38.003 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-11-25T09:10:38.004 INFO:teuthology.task.ansible.out:fatal: [smithi081.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.039367", "end": "2019-11-25 09:10:37.969879", "msg": "non-zero return code", "rc": 1, "start": "2019-11-25 09:10:37.930512", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-11-25T09:10:38.005 INFO:teuthology.task.ansible.out: 2019-11-25T09:10:38.046 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:38 +0000 (0:00:00.403) 0:03:09.427 ******* 2019-11-25T09:10:38.083 INFO:teuthology.task.ansible.out: TASK [testnode : Removing saved yum transactions] ****************************** 2019-11-25T09:10:38.084 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "cmd": ["yum-complete-transaction", "--cleanup-only"], "delta": "0:00:00.293961", "end": "2019-11-25 09:10:38.039442", "rc": 0, "start": "2019-11-25 09:10:37.745481", "stderr": "", "stderr_lines": [], "stdout": "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: packages.oit.ncsu.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.jaleco.com\n * updates: mirror.trouble-free.net\nNo unfinished transactions left.", "stdout_lines": ["Loaded plugins: fastestmirror, langpacks, priorities", "Loading mirror speeds from cached hostfile", " * base: packages.oit.ncsu.edu", " * epel: download-cc-rdu01.fedoraproject.org", " * extras: mirror.jaleco.com", " * updates: mirror.trouble-free.net", "No unfinished transactions left."]} 2019-11-25T09:10:38.085 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:38.124 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:38 +0000 (0:00:00.078) 0:03:09.505 ******* 2019-11-25T09:10:38.164 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:38 +0000 (0:00:00.039) 0:03:09.545 ******* 2019-11-25T09:10:38.545 INFO:teuthology.task.ansible.out: TASK [testnode : Check if ceph-debuginfo is installed] ************************* 2019-11-25T09:10:38.546 INFO:teuthology.task.ansible.out:fatal: [smithi153.front.sepia.ceph.com]: FAILED! => {"changed": false, "cmd": ["rpm", "-q", "ceph-debuginfo"], "delta": "0:00:00.053841", "end": "2019-11-25 09:10:38.511085", "msg": "non-zero return code", "rc": 1, "start": "2019-11-25 09:10:38.457244", "stderr": "", "stderr_lines": [], "stdout": "package ceph-debuginfo is not installed", "stdout_lines": ["package ceph-debuginfo is not installed"]} ...ignoring 2019-11-25T09:10:38.589 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:38 +0000 (0:00:00.425) 0:03:09.970 ******* 2019-11-25T09:10:38.636 INFO:teuthology.task.ansible.out: TASK [testnode : Work around https://bugzilla.redhat.com/show_bug.cgi?id=1234967] *** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:10:38.672 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:38 +0000 (0:00:00.083) 0:03:10.053 ******* 2019-11-25T09:10:48.605 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 2019-11-25T09:10:48.606 INFO:teuthology.task.ansible.out: @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemov 2019-11-25T09:10:48.608 INFO:teuthology.task.ansible.out:ed:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-11-25T09:10:48.635 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:48 +0000 (0:00:09.962) 0:03:20.016 ******* 2019-11-25T09:10:49.280 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["librbd1", "librados2", "mod_fastcgi", "iozone"]}, "msg": "", "rc": 0, "results": ["ceph is not installed", "ceph-base is not installed", "ceph-selinux is not installed", "ceph-common is not installed", "ceph-debuginfo is not installed", "ceph-release is not installed", "libcephfs1 is not installed", "ceph-radosgw is not installed", "python-ceph is not installed", "python-rados is not installed", "python-rbd is not installed", "python-cephfs is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be erased\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be erased\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-img-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be erased\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be erased\n--> Running transaction check\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be erased\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n iozone x86_64 3.424-2_ceph.el7.centos @lab-extras 1.3 M\n librados2 x86_64 1:10.2.5-4.el7 @base 5.5 M\n librbd1 x86_64 1:10.2.5-4.el7 2019-11-25T09:10:49.282 INFO:teuthology.task.ansible.out: @base 7.4 M\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos @centos7-fcgi-ceph 213 k\nRemoving for dependencies:\n qemu-img x86_64 10:1.5.3-167.el7_7.1 @updates 2.3 M\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 @updates 5.7 M\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 @updates 755 k\n\nTransaction Summary\n================================================================================\nRemove 4 Packages (+3 Dependent packages)\n\nInstalled size: 23 M\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 1/7 \n Erasing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 2/7 \n Erasing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 3/7 \n Erasing : 1:librbd1-10.2.5-4.el7.x86_64 4/7 \n Erasing : 1:librados2-10.2.5-4.el7.x86_64 5/7 \n Erasing : iozone-3.424-2_ceph.el7.centos.x86_64 6/7 \n Erasing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 7/7 \nwarning: file /var/run/mod_fastcgi: remove failed: No such file or directory\n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/7 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/7 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/7 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/7 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/7 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/7 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 7/7 \n\nRemoved:\n iozone.x86_64 0:3.424-2_ceph.el7.centos 2019-11-25T09:10:49.283 INFO:teuthology.task.ansible.out: \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n\nDependency Removed:\n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-11-25T09:10:49.321 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:49 +0000 (0:00:00.686) 0:03:20.702 ******* 2019-11-25T09:10:51.977 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-11-25T09:10:51.978 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-11-25T09:10:52.015 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:52 +0000 (0:00:02.693) 0:03:23.395 ******* 2019-11-25T09:10:52.779 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "changes": {"removed": ["boost-random", "boost-random"]}, "msg": "", "rc": 0, "results": ["boost-program-options is not installed", "leveldb is not installed", "xmlstarlet is not installed", "python-jinja2 is not installed", "python-ceph is not installed", "python-flask is not installed", "python-requests is not installed", "python-urllib3 is not installed", "python-babel is not installed", "hdparm is not installed", "python-markupsafe is not installed", "python-werkzeug is not installed", "python-itsdangerous is not installed", "Loaded plugins: fastestmirror, langpacks, priorities\nResolving Dependencies\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be erased\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nRemoving:\n boost-random x86_64 1.53.0-27.el7 @base 28 k\n\nTransaction Summary\n================================================================================\nRemove 1 Package\n\nInstalled size: 28 k\nDownloading packages:\nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Erasing : boost-random-1.53.0-27.el7.x86_64 1/1 \n Verifying : boost-random-1.53.0-27.el7.x86_64 1/1 \n\nRemoved:\n boost-random.x86_64 0:1.53.0-27.el7 \n\nComplete!\n"]} 2019-11-25T09:10:52.818 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:10:52 +0000 (0:00:00.803) 0:03:24.199 ******* 2019-11-25T09:11:09.933 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid i 2019-11-25T09:11:09.935 INFO:teuthology.task.ansible.out:s already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is already installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel 2019-11-25T09:11:09.943 INFO:teuthology.task.ansible.out:-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already installed", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: packages.oit.ncsu.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.jaleco.com\n * updates: mirror.atlanticmetro.net\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.5 2019-11-25T09:11:09.945 INFO:teuthology.task.ansible.out:3.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Resolved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 18 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 2019-11-25T09:11:09.947 INFO:teuthology.task.ansible.out: 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-11-25T09:11:09.967 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:09 +0000 (0:00:17.148) 0:03:41.348 ******* 2019-11-25T09:11:10.757 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** 2019-11-25T09:11:10.758 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-11-25T09:11:10.796 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:10 +0000 (0:00:00.829) 0:03:42.177 ******* 2019-11-25T09:11:11.333 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-11-25T09:11:11.334 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-11-25T09:11:11.369 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:11 +0000 (0:00:00.573) 0:03:42.750 ******* 2019-11-25T09:11:11.406 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-25T09:11:11.407 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Install packages] ********************************************* 2019-11-25T09:11:11.409 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "changes": {"installed": ["@core", "@base", "iozone", "qemu-kvm", "mod_fastcgi-2.4.7-1.ceph.el7.centos"]}, "msg": "Warning: Group core does not have any packages to install.\nWarning: Group base does not have any packages to install.\nGroup base does have 1 conditional packages, which may get installed.\n", "rc": 0, "results": ["yum-plugin-priorities-1.1.31-52.el7.noarch providing yum-plugin-priorities is already installed", "yum-plugin-fastestmirror-1.1.31-50.el7.noarch providing yum-plugin-fastestmirror is already installed", "redhat-lsb-4.1-27.el7.centos.1.x86_64 providing redhat-lsb is already installed", "sysstat-10.1.5-17.el7.x86_64 providing sysstat is already installed", "gdb-7.6.1-114.el7.x86_64 providing gdb is already installed", "git-all-1.8.3.1-20.el7.noarch providing git-all is already installed", "python-configobj-4.7.2-7.el7.noarch providing python-configobj is already installed", "gcc-c++-4.8.5-39.el7.x86_64 providing gcc-c++ is already installed", "libedit-3.0-12.20121213cvs.el7.x86_64 providing libedit is already installed", "openssl098e-0.9.8e-29.el7.centos.3.x86_64 providing openssl098e is already installed", "boost-thread-1.53.0-27.el7.x86_64 providing boost-thread is already installed", "xfsprogs-4.5.0-20.el7.x86_64 providing xfsprogs is already installed", "gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "parted-3.1-29.el7.x86_64 providing parted is already installed", "libgcrypt-1.5.3-14.el7.x86_64 providing libgcrypt is already installed", "fuse-2.9.2-11.el7.x86_64 providing fuse is already installed", "fuse-libs-2.9.2-11.el7.x86_64 providing fuse-libs is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "1:openssl-1.0.2k-16.el7.x86_64 providing openssl is already installed", "libuuid-2.23.2-61.el7.x86_64 providing libuuid is already installed", "btrfs-progs-4.9.1-1.el7.x86_64 providing btrfs-progs is alrea 2019-11-25T09:11:11.410 INFO:teuthology.task.ansible.out:dy installed", "attr-2.4.46-13.el7.x86_64 providing attr is already installed", "1:valgrind-3.14.0-16.el7.x86_64 providing valgrind is already installed", "python-nose-1.3.7-1.el7.noarch providing python-nose is already installed", "mpich-3.0-3.0.4-10.el7.x86_64 providing mpich is already installed", "podman-1.4.4-4.el7.centos.x86_64 providing podman is already installed", "ant-1.9.4-2.el7.noarch providing ant is already installed", "libtool-2.4.2-22.el7_3.x86_64 providing libtool is already installed", "automake-1.13.4-3.el7.noarch providing automake is already installed", "gettext-0.19.8.1-2.el7.x86_64 providing gettext is already installed", "libuuid-devel-2.23.2-61.el7.x86_64 providing libuuid-devel is already installed", "libacl-devel-2.2.51-14.el7.x86_64 providing libacl-devel is already installed", "bc-1.06.95-13.el7.x86_64 providing bc is already installed", "xfsdump-3.1.7-1.el7.x86_64 providing xfsdump is already installed", "xfsprogs-devel-4.5.0-20.el7.x86_64 providing xfsprogs-devel is already installed", "blktrace-1.0.5-8.el7.x86_64 providing blktrace is already installed", "1:numpy-1.7.1-13.el7.x86_64 providing numpy is already installed", "python-matplotlib-1.2.0-16.el7.x86_64 providing python-matplotlib is already installed", "usbredir-0.7.1-3.el7.x86_64 providing usbredir is already installed", "genisoimage-1.1.11-25.el7.x86_64 providing genisoimage is already installed", "httpd-2.4.6-90.el7.centos.x86_64 providing httpd is already installed", "httpd-devel-2.4.6-90.el7.centos.x86_64 providing httpd-devel is already installed", "httpd-tools-2.4.6-90.el7.centos.x86_64 providing httpd-tools is already installed", "1:mod_ssl-2.4.6-90.el7.centos.x86_64 providing mod_ssl is already installed", "libev-devel-4.15-7.el7.x86_64 providing libev-devel is already installed", "perl-XML-Twig-3.44-2.el7.noarch providing perl-XML-Twig is already installed", "1:java-1.6.0-openjdk-devel-1.6.0.41-1.13.13.1.el7_3.x86_64 providing java-1.6.0-openjdk-devel is already insta 2019-11-25T09:11:11.417 INFO:teuthology.task.ansible.out:lled", "junit-4.11-8.el7.noarch providing junit4 is already installed", "1:nfs-utils-1.3.0-0.65.el7.x86_64 providing nfs-utils is already installed", "ncurses-devel-5.9-14.20130511.el7_4.x86_64 providing ncurses-devel is already installed", "python-devel-2.7.5-86.el7.x86_64 providing python-devel is already installed", "python-virtualenv-15.1.0-2.el7.noarch providing python-virtualenv is already installed", "perl-CPAN-1.9800-294.el7_6.noarch providing perl-CPAN is already installed", "Loaded plugins: fastestmirror, langpacks, priorities\nLoading mirror speeds from cached hostfile\n * base: packages.oit.ncsu.edu\n * epel: download-cc-rdu01.fedoraproject.org\n * extras: mirror.jaleco.com\n * updates: mirror.trouble-free.net\nResolving Dependencies\n--> Running transaction check\n---> Package iozone.x86_64 0:3.424-2_ceph.el7.centos will be installed\n---> Package mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos will be installed\n---> Package qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Processing Dependency: qemu-kvm-common = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: qemu-img = 10:1.5.3-167.el7_7.1 for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librbd.so.1()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Processing Dependency: librados.so.2()(64bit) for package: 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64\n--> Running transaction check\n---> Package librados2.x86_64 1:10.2.5-4.el7 will be installed\n--> Processing Dependency: libboost_random-mt.so.1.53.0()(64bit) for package: 1:librados2-10.2.5-4.el7.x86_64\n---> Package librbd1.x86_64 1:10.2.5-4.el7 will be installed\n---> Package qemu-img.x86_64 10:1.5.3-167.el7_7.1 will be installed\n---> Package qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 will be installed\n--> Running transaction check\n---> Package boost-random.x86_64 0:1.53.0-27.el7 will be installed\n--> Finished Dependency Resolution\n\nDependencies Reso 2019-11-25T09:11:11.419 INFO:teuthology.task.ansible.out:lved\n\n================================================================================\n Package Arch Version Repository Size\n================================================================================\nInstalling:\n iozone x86_64 3.424-2_ceph.el7.centos lab-extras 800 k\n mod_fastcgi x86_64 2.4.7-1.ceph.el7.centos centos7-fcgi-ceph 70 k\n qemu-kvm x86_64 10:1.5.3-167.el7_7.1 updates 1.9 M\nInstalling for dependencies:\n boost-random x86_64 1.53.0-27.el7 base 39 k\n librados2 x86_64 1:10.2.5-4.el7 base 1.8 M\n librbd1 x86_64 1:10.2.5-4.el7 base 2.4 M\n qemu-img x86_64 10:1.5.3-167.el7_7.1 updates 699 k\n qemu-kvm-common x86_64 10:1.5.3-167.el7_7.1 updates 435 k\n\nTransaction Summary\n================================================================================\nInstall 3 Packages (+5 Dependent packages)\n\nTotal download size: 8.1 M\nInstalled size: 23 M\nDownloading packages:\n--------------------------------------------------------------------------------\nTotal 18 MB/s | 8.1 MB 00:00 \nRunning transaction check\nRunning transaction test\nTransaction test succeeded\nRunning transaction\n Installing : boost-random-1.53.0-27.el7.x86_64 1/8 \n Installing : 1:librados2-10.2.5-4.el7.x86_64 2/8 \n Installing : 1:librbd1-10.2.5-4.el7.x86_64 3/8 \n Installing : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 4/8 \n Installing : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 5/8 \n Installing : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 6/8 \n Installing : iozone-3.424-2_ceph.el7.centos.x86_64 2019-11-25T09:11:11.421 INFO:teuthology.task.ansible.out: 7/8 \n Installing : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 8/8 \n Verifying : 1:librbd1-10.2.5-4.el7.x86_64 1/8 \n Verifying : 10:qemu-kvm-common-1.5.3-167.el7_7.1.x86_64 2/8 \n Verifying : mod_fastcgi-2.4.7-1.ceph.el7.centos.x86_64 3/8 \n Verifying : 1:librados2-10.2.5-4.el7.x86_64 4/8 \n Verifying : 10:qemu-kvm-1.5.3-167.el7_7.1.x86_64 5/8 \n Verifying : 10:qemu-img-1.5.3-167.el7_7.1.x86_64 6/8 \n Verifying : boost-random-1.53.0-27.el7.x86_64 7/8 \n Verifying : iozone-3.424-2_ceph.el7.centos.x86_64 8/8 \n\nInstalled:\n iozone.x86_64 0:3.424-2_ceph.el7.centos \n mod_fastcgi.x86_64 0:2.4.7-1.ceph.el7.centos \n qemu-kvm.x86_64 10:1.5.3-167.el7_7.1 \n\nDependency Installed:\n boost-random.x86_64 0:1.53.0-27.el7 \n librados2.x86_64 1:10.2.5-4.el7 \n librbd1.x86_64 1:10.2.5-4.el7 \n qemu-img.x86_64 10:1.5.3-167.el7_7.1 \n qemu-kvm-common.x86_64 10:1.5.3-167.el7_7.1 \n\nComplete!\n"]} 2019-11-25T09:11:11.443 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:11 +0000 (0:00:00.074) 0:03:42.824 ******* 2019-11-25T09:11:11.477 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:11 +0000 (0:00:00.033) 0:03:42.858 ******* 2019-11-25T09:11:11.865 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-11-25T09:11:11.867 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service dbus.service basic.target system.slice", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "network-pre.target shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ebtables.service iptables.service ipset.service ip6tables.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-11-25T09:11:11.868 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-11-25T09:11:11.870 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "network-pre.target system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-11-25T09:11:11.915 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:11 +0000 (0:00:00.437) 0:03:43.295 ******* 2019-11-25T09:11:12.261 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi081.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-11-25T09:11:12.304 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:12 +0000 (0:00:00.385) 0:03:43.681 ******* 2019-11-25T09:11:12.309 INFO:teuthology.task.ansible.out: TASK [testnode : Install epel packages] **************************************** ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["cryptopp-devel-5.6.2-10.el7.x86_64 providing cryptopp-devel is already installed", "cryptopp-5.6.2-10.el7.x86_64 providing cryptopp is already installed", "fcgi-2.4.0-25.el7.x86_64 providing fcgi is already installed", "dbench-4.0-10.el7.x86_64 providing dbench is already installed", "fuse-sshfs-2.10-1.el7.x86_64 providing fuse-sshfs is already installed", "bonnie++-1.97.3-1.el7.x86_64 providing bonnie++ is already installed", "1:perl-JSON-XS-3.01-2.el7.x86_64 providing perl-JSON-XS is already installed"]} 2019-11-25T09:11:12.354 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:12 +0000 (0:00:00.051) 0:03:43.733 ******* 2019-11-25T09:11:12.914 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-11-25T09:11:12.916 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["device-mapper-multipath is not installed"]} 2019-11-25T09:11:12.958 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:12 +0000 (0:00:00.605) 0:03:44.338 ******* 2019-11-25T09:11:12.996 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.031 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.073) 0:03:44.412 ******* 2019-11-25T09:11:13.131 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-11-25T09:11:13.133 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-11-25T09:11:13.181 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.149) 0:03:44.562 ******* 2019-11-25T09:11:13.230 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-11-25T09:11:13.232 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.274 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.093) 0:03:44.655 ******* 2019-11-25T09:11:13.318 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-11-25T09:11:13.356 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.081) 0:03:44.737 ******* 2019-11-25T09:11:13.382 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-11-25T09:11:13.384 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.419 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.063) 0:03:44.800 ******* 2019-11-25T09:11:13.446 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-11-25T09:11:13.448 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.487 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.067) 0:03:44.868 ******* 2019-11-25T09:11:13.495 INFO:teuthology.task.ansible.out: TASK [testnode : Stop and disable firewalld] *********************************** 2019-11-25T09:11:13.499 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "enabled": false, "name": "firewalld", "state": "stopped", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "polkit.service system.slice basic.target dbus.service", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target network-pre.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "BusName": "org.fedoraproject.FirewallD1", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "yes", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "Conflicts": "ipset.service ip6tables.service ebtables.service iptables.service shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "firewalld - dynamic firewall daemon", "DevicePolicy": "auto", "Documentation": "man:firewalld(1)", "EnvironmentFile": "/etc/sysconfig/firewalld (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecReload": "{ path=/bin/kill ; argv[]=/bin/kill -HUP $MAINPID ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "ExecStart": "{ path=/usr/sbin/firewalld ; argv[]=/usr/sbin/firewalld --nofork --nopid $FIREWALLD_ARGS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/firewalld.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "firewalld.service", "IgnoreOnIsolate": "no", "Igno 2019-11-25T09:11:13.500 INFO:teuthology.task.ansible.out:reOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "mixed", "KillSignal": "15", "LimitAS": "18446744073709551615", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "firewalld.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "no", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "null", "StandardInput": "null", "StandardOutput": "null", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", " 2019-11-25T09:11:13.502 INFO:teuthology.task.ansible.out:TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec": "50000", "Transient": "no", "Type": "dbus", "UMask": "0022", "UnitFilePreset": "enabled", "UnitFileState": "disabled", "Wants": "system.slice network-pre.target", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-11-25T09:11:13.543 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.055) 0:03:44.923 ******* 2019-11-25T09:11:13.549 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-11-25T09:11:13.550 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.551 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.589 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.046) 0:03:44.970 ******* 2019-11-25T09:11:13.616 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-11-25T09:11:13.617 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.653 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.063) 0:03:45.034 ******* 2019-11-25T09:11:13.681 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-11-25T09:11:13.716 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.063) 0:03:45.097 ******* 2019-11-25T09:11:13.742 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-11-25T09:11:13.744 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.782 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.065) 0:03:45.163 ******* 2019-11-25T09:11:13.808 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-11-25T09:11:13.810 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.845 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.063) 0:03:45.226 ******* 2019-11-25T09:11:13.872 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-11-25T09:11:13.873 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:13.908 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.062) 0:03:45.289 ******* 2019-11-25T09:11:13.935 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-25T09:11:13.937 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Stop and disable iptables] ************************************ fatal: [smithi153.front.sepia.ceph.com]: FAILED! => {"changed": false, "msg": "Could not find the requested service iptables: host"} ...ignoring 2019-11-25T09:11:13.975 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:13 +0000 (0:00:00.066) 0:03:45.355 ******* 2019-11-25T09:11:14.008 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.033) 0:03:45.389 ******* 2019-11-25T09:11:14.013 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade non aarch64 packages] ********************************* 2019-11-25T09:11:14.014 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.054 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.046) 0:03:45.435 ******* 2019-11-25T09:11:14.094 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.133 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.078) 0:03:45.514 ******* 2019-11-25T09:11:14.159 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-11-25T09:11:14.161 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.198 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.065) 0:03:45.579 ******* 2019-11-25T09:11:14.225 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** 2019-11-25T09:11:14.227 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.262 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.063) 0:03:45.642 ******* 2019-11-25T09:11:14.288 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-11-25T09:11:14.289 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.325 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.063) 0:03:45.706 ******* 2019-11-25T09:11:14.351 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ 2019-11-25T09:11:14.353 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.389 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.063) 0:03:45.769 ******* 2019-11-25T09:11:14.394 INFO:teuthology.task.ansible.out: TASK [testnode : Enable SELinux] *********************************************** 2019-11-25T09:11:14.395 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "configfile": "/etc/selinux/config", "msg": "", "policy": "targeted", "reboot_required": false, "state": "permissive"} 2019-11-25T09:11:14.432 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.043) 0:03:45.813 ******* 2019-11-25T09:11:14.438 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** 2019-11-25T09:11:14.438 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.475 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.042) 0:03:45.856 ******* 2019-11-25T09:11:14.480 INFO:teuthology.task.ansible.out: TASK [testnode : Check for custom repos] *************************************** 2019-11-25T09:11:14.482 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.518 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.043) 0:03:45.899 ******* 2019-11-25T09:11:14.524 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-11-25T09:11:14.525 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.561 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.042) 0:03:45.942 ******* 2019-11-25T09:11:14.567 INFO:teuthology.task.ansible.out: TASK [testnode : Remove custom repos] ****************************************** 2019-11-25T09:11:14.604 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.043) 0:03:45.985 ******* 2019-11-25T09:11:14.612 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-11-25T09:11:14.614 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.648 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.044) 0:03:46.029 ******* 2019-11-25T09:11:14.654 INFO:teuthology.task.ansible.out: TASK [testnode : Set apt preferences] ****************************************** 2019-11-25T09:11:14.656 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.694 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.044) 0:03:46.073 ******* 2019-11-25T09:11:14.698 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-11-25T09:11:14.700 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.747 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.054) 0:03:46.128 ******* 2019-11-25T09:11:14.753 INFO:teuthology.task.ansible.out: TASK [testnode : Add sources list] ********************************************* 2019-11-25T09:11:14.755 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.795 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.048) 0:03:46.176 ******* 2019-11-25T09:11:14.800 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-11-25T09:11:14.801 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.840 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.044) 0:03:46.221 ******* 2019-11-25T09:11:14.849 INFO:teuthology.task.ansible.out: TASK [testnode : Install apt keys] ********************************************* 2019-11-25T09:11:14.851 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/autobuild.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/autobuild.asc", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.851 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=http://download.ceph.com/keys/release.asc) => {"ansible_loop_var": "item", "changed": false, "item": "http://download.ceph.com/keys/release.asc", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.889 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.049) 0:03:46.270 ******* 2019-11-25T09:11:14.898 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.934 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.044) 0:03:46.314 ******* 2019-11-25T09:11:14.940 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-apt] ******************************************* 2019-11-25T09:11:14.941 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:14.978 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:14 +0000 (0:00:00.044) 0:03:46.359 ******* 2019-11-25T09:11:14.986 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* 2019-11-25T09:11:14.987 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.024 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.045) 0:03:46.405 ******* 2019-11-25T09:11:15.030 INFO:teuthology.task.ansible.out: TASK [testnode : Add local apt repos.] ***************************************** 2019-11-25T09:11:15.067 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.043) 0:03:46.448 ******* 2019-11-25T09:11:15.074 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-11-25T09:11:15.076 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.111 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.043) 0:03:46.492 ******* 2019-11-25T09:11:15.117 INFO:teuthology.task.ansible.out: TASK [testnode : Blast the apt cache.] ***************************************** 2019-11-25T09:11:15.119 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.165 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.044) 0:03:46.536 ******* TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.206 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.050) 0:03:46.587 ******* 2019-11-25T09:11:15.213 INFO:teuthology.task.ansible.out: TASK [testnode : Update apt cache.] ******************************************** 2019-11-25T09:11:15.214 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.251 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.044) 0:03:46.632 ******* 2019-11-25T09:11:15.268 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.294 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.043) 0:03:46.675 ******* 2019-11-25T09:11:15.303 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure packages are not present.] ***************************** 2019-11-25T09:11:15.304 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.339 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.044) 0:03:46.720 ******* 2019-11-25T09:11:15.346 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.383 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.044) 0:03:46.764 ******* 2019-11-25T09:11:15.401 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.441 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.058) 0:03:46.822 ******* 2019-11-25T09:11:15.450 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-11-25T09:11:15.452 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.504 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.054) 0:03:46.876 ******* TASK [testnode : Upgrade non aarch64 packages] ********************************* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.553 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.057) 0:03:46.934 ******* 2019-11-25T09:11:15.568 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.600 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.046) 0:03:46.980 ******* 2019-11-25T09:11:15.606 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-11-25T09:11:15.607 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.646 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.046) 0:03:47.027 ******* 2019-11-25T09:11:15.655 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.690 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.044) 0:03:47.071 ******* 2019-11-25T09:11:15.697 INFO:teuthology.task.ansible.out: TASK [testnode : Install non aarch64 packages] ********************************* 2019-11-25T09:11:15.698 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.735 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.044) 0:03:47.116 ******* 2019-11-25T09:11:15.741 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-25T09:11:15.742 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.779 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.043) 0:03:47.160 ******* 2019-11-25T09:11:15.800 INFO:teuthology.task.ansible.out: TASK [testnode : Use perl to add user_xattr to the root mount options in fstab.] *** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.831 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.049) 0:03:47.210 ******* 2019-11-25T09:11:15.836 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.873 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.043) 0:03:47.253 ******* 2019-11-25T09:11:15.878 INFO:teuthology.task.ansible.out: TASK [testnode : Enable xattr for this boot.] ********************************** 2019-11-25T09:11:15.880 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.916 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.043) 0:03:47.297 ******* 2019-11-25T09:11:15.949 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure fuse group exists.] ************************************ skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:15.978 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:15 +0000 (0:00:00.062) 0:03:47.359 ******* 2019-11-25T09:11:16.015 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/fuse.conf.] *************************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.061 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.082) 0:03:47.441 ******* 2019-11-25T09:11:16.087 INFO:teuthology.task.ansible.out: TASK [testnode : Add teuthology user to groups fuse, kvm and disk.] ************ 2019-11-25T09:11:16.088 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.125 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.064) 0:03:47.506 ******* 2019-11-25T09:11:16.162 INFO:teuthology.task.ansible.out: TASK [testnode : Set up static IP in /etc/hosts.] ****************************** 2019-11-25T09:11:16.164 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.210 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.085) 0:03:47.591 ******* 2019-11-25T09:11:16.217 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-11-25T09:11:16.219 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-11-25T09:11:16.264 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.053) 0:03:47.645 ******* 2019-11-25T09:11:16.273 INFO:teuthology.task.ansible.out: TASK [testnode : Stop apache2] ************************************************* 2019-11-25T09:11:16.275 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.312 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.048) 0:03:47.693 ******* 2019-11-25T09:11:16.339 INFO:teuthology.task.ansible.out: TASK [testnode : Set mode on /etc/fuse.conf] *********************************** 2019-11-25T09:11:16.341 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.386 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.073) 0:03:47.766 ******* 2019-11-25T09:11:16.413 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure the group kvm exists.] ********************************* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.450 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.064) 0:03:47.830 ******* 2019-11-25T09:11:16.478 INFO:teuthology.task.ansible.out: TASK [testnode : Add the teuthology user to groups kvm,disk] ******************* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.514 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.063) 0:03:47.894 ******* 2019-11-25T09:11:16.539 INFO:teuthology.task.ansible.out: TASK [testnode : Configure /etc/sudoers.] ************************************** 2019-11-25T09:11:16.541 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.577 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.063) 0:03:47.958 ******* 2019-11-25T09:11:16.603 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-11-25T09:11:16.605 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} TASK [testnode : Configure /etc/security/limits.conf] ************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.652 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.074) 0:03:48.032 ******* 2019-11-25T09:11:16.686 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.034) 0:03:48.067 ******* 2019-11-25T09:11:16.722 INFO:teuthology.task.ansible.out: TASK [testnode : Rebuild rpmdb] ************************************************ 2019-11-25T09:11:16.724 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.771 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.084) 0:03:48.152 ******* 2019-11-25T09:11:16.797 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph packages are not present.] ************************ 2019-11-25T09:11:16.799 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.835 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.064) 0:03:48.216 ******* 2019-11-25T09:11:16.861 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure ceph dependency packages are not present.] ************* 2019-11-25T09:11:16.863 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.903 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.067) 0:03:48.284 ******* 2019-11-25T09:11:16.929 INFO:teuthology.task.ansible.out: TASK [testnode : Remove packages] ********************************************** 2019-11-25T09:11:16.931 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:16.973 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:16 +0000 (0:00:00.070) 0:03:48.354 ******* 2019-11-25T09:11:16.990 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-11-25T09:11:16.997 INFO:teuthology.task.ansible.out: [WARNING]: Cannot set fs attributes on a non-existent symlink target. follow should be set to False to avoid this. ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-11-25T09:11:17.031 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.056) 0:03:48.411 ******* 2019-11-25T09:11:17.038 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages] ********************************************* 2019-11-25T09:11:17.040 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.077 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.047) 0:03:48.458 ******* 2019-11-25T09:11:17.082 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-11-25T09:11:17.083 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.137 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.060) 0:03:48.518 ******* 2019-11-25T09:11:17.143 INFO:teuthology.task.ansible.out: TASK [testnode : Upgrade packages] ********************************************* 2019-11-25T09:11:17.145 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.188 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.050) 0:03:48.569 ******* 2019-11-25T09:11:17.196 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-11-25T09:11:17.197 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.238 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.049) 0:03:48.619 ******* 2019-11-25T09:11:17.245 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-11-25T09:11:17.247 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.283 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.044) 0:03:48.663 ******* 2019-11-25T09:11:17.289 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.342 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.058) 0:03:48.722 ******* 2019-11-25T09:11:17.368 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.410 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.068) 0:03:48.791 ******* 2019-11-25T09:11:17.436 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-11-25T09:11:17.437 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.476 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.065) 0:03:48.857 ******* 2019-11-25T09:11:17.503 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-11-25T09:11:17.504 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.540 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.063) 0:03:48.921 ******* 2019-11-25T09:11:17.566 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-11-25T09:11:17.568 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.604 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.063) 0:03:48.985 ******* 2019-11-25T09:11:17.609 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-11-25T09:11:17.610 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/bin/lspci", "gid": 0, "group": "root", "mode": "0777", "owner": "root", "secontext": "unconfined_u:object_r:bin_t:s0", "size": 11, "src": "/sbin/lspci", "state": "link", "uid": 0} 2019-11-25T09:11:17.658 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.048) 0:03:49.033 ******* TASK [testnode : Create FStest ubuntu directory.] ****************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.698 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.045) 0:03:49.078 ******* 2019-11-25T09:11:17.739 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.766 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.066) 0:03:49.145 ******* 2019-11-25T09:11:17.792 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-11-25T09:11:17.794 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.838 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.070) 0:03:49.216 ******* 2019-11-25T09:11:17.863 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.900 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.064) 0:03:49.280 ******* 2019-11-25T09:11:17.926 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:17.968 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:17 +0000 (0:00:00.065) 0:03:49.346 ******* 2019-11-25T09:11:17.983 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/usr/lib/ltp/testcases/bin", "secontext": "unconfined_u:object_r:lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-25T09:11:18.026 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.061) 0:03:49.407 ******* 2019-11-25T09:11:18.032 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-11-25T09:11:18.034 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.074 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.047) 0:03:49.454 ******* 2019-11-25T09:11:18.101 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-11-25T09:11:18.102 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.139 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.065) 0:03:49.520 ******* 2019-11-25T09:11:18.166 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-11-25T09:11:18.168 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.208 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.065) 0:03:49.585 ******* 2019-11-25T09:11:18.232 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** 2019-11-25T09:11:18.234 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.273 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.068) 0:03:49.654 ******* 2019-11-25T09:11:18.275 INFO:teuthology.task.ansible.out: 2019-11-25T09:11:18.301 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.337 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.063) 0:03:49.718 ******* 2019-11-25T09:11:18.363 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-11-25T09:11:18.365 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.404 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.066) 0:03:49.784 ******* 2019-11-25T09:11:18.409 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-11-25T09:11:18.410 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "dest": "/usr/lib/ltp/testcases/bin/fsstress", "src": "/usr/bin/fsstress"} 2019-11-25T09:11:18.449 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.045) 0:03:49.830 ******* 2019-11-25T09:11:18.454 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-11-25T09:11:18.456 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.493 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.043) 0:03:49.874 ******* 2019-11-25T09:11:18.499 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-11-25T09:11:18.503 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.548 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.053) 0:03:49.927 ******* 2019-11-25T09:11:18.554 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-11-25T09:11:18.554 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.594 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.047) 0:03:49.974 ******* 2019-11-25T09:11:18.600 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-11-25T09:11:18.601 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.643 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.049) 0:03:50.024 ******* 2019-11-25T09:11:18.649 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-11-25T09:11:18.651 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.688 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.044) 0:03:50.068 ******* 2019-11-25T09:11:18.694 INFO:teuthology.task.ansible.out: TASK [testnode : Include hostname in /etc/sysconfig/network] ******************* 2019-11-25T09:11:18.696 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.744 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.056) 0:03:50.125 ******* 2019-11-25T09:11:18.750 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-11-25T09:11:18.751 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.789 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.044) 0:03:50.169 ******* 2019-11-25T09:11:18.794 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-11-25T09:11:18.796 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.834 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.046) 0:03:50.215 ******* 2019-11-25T09:11:18.840 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:11:18.841 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-11-25T09:11:18.879 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.044) 0:03:50.260 ******* 2019-11-25T09:11:18.885 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-11-25T09:11:18.886 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.925 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.046) 0:03:50.306 ******* 2019-11-25T09:11:18.931 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-11-25T09:11:18.931 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.944 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} skipping: [smithi081.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.946 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.983 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:18 +0000 (0:00:00.058) 0:03:50.364 ******* 2019-11-25T09:11:18.987 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** 2019-11-25T09:11:18.988 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:18.990 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.032 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.048) 0:03:50.413 ******* 2019-11-25T09:11:19.038 INFO:teuthology.task.ansible.out: TASK [testnode : Make raid/smart scripts work.] ******************************** 2019-11-25T09:11:19.040 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.079 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.044) 0:03:50.457 ******* 2019-11-25T09:11:19.103 INFO:teuthology.task.ansible.out: TASK [testnode : Create FStest ubuntu directory.] ****************************** 2019-11-25T09:11:19.107 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.140 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.063) 0:03:50.521 ******* 2019-11-25T09:11:19.168 INFO:teuthology.task.ansible.out: TASK [testnode : Make fsstress same path as ubuntu.] *************************** 2019-11-25T09:11:19.169 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.205 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.065) 0:03:50.586 ******* 2019-11-25T09:11:19.211 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-11-25T09:11:19.212 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003855", "end": "2019-11-25 09:11:19.155467", "rc": 0, "start": "2019-11-25 09:11:19.151612", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-11-25T09:11:19.261 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.054) 0:03:50.640 ******* 2019-11-25T09:11:19.265 INFO:teuthology.task.ansible.out: TASK [testnode : Set grub config.] ********************************************* 2019-11-25T09:11:19.268 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.310 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.049) 0:03:50.690 ******* 2019-11-25T09:11:19.315 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-11-25T09:11:19.317 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.359 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.049) 0:03:50.740 ******* 2019-11-25T09:11:19.365 INFO:teuthology.task.ansible.out: TASK [testnode : Remove /etc/ceph] ********************************************* 2019-11-25T09:11:19.366 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.407 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.048) 0:03:50.788 ******* 2019-11-25T09:11:19.413 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-11-25T09:11:19.414 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.453 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.045) 0:03:50.834 ******* 2019-11-25T09:11:19.460 INFO:teuthology.task.ansible.out: TASK [testnode : Upload a dummy nfs export so that the nfs kernel server starts.] *** 2019-11-25T09:11:19.461 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.499 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.045) 0:03:50.880 ******* 2019-11-25T09:11:19.505 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-11-25T09:11:19.506 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.543 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.044) 0:03:50.924 ******* 2019-11-25T09:11:19.550 INFO:teuthology.task.ansible.out: TASK [testnode : Enable nfs-server on rhel 7.x.] ******************************* 2019-11-25T09:11:19.551 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.588 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.044) 0:03:50.969 ******* 2019-11-25T09:11:19.593 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-11-25T09:11:19.595 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.632 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.043) 0:03:51.012 ******* 2019-11-25T09:11:19.637 INFO:teuthology.task.ansible.out: TASK [testnode : Upload weekly kernel-clean crontab.] ************************** 2019-11-25T09:11:19.639 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.675 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.043) 0:03:51.056 ******* 2019-11-25T09:11:19.699 INFO:teuthology.task.ansible.out: TASK [testnode : Upload /etc/grub.d/02_force_timeout.] ************************* 2019-11-25T09:11:19.701 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.736 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.060) 0:03:51.117 ******* 2019-11-25T09:11:19.777 INFO:teuthology.task.ansible.out: TASK [testnode : Enable kernel modules to load at boot time.] ****************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.809 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.073) 0:03:51.190 ******* 2019-11-25T09:11:19.835 INFO:teuthology.task.ansible.out: TASK [testnode : Enabling auto-fsck fix to prevent boot hangup.] *************** 2019-11-25T09:11:19.837 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.873 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.063) 0:03:51.254 ******* 2019-11-25T09:11:19.899 INFO:teuthology.task.ansible.out: TASK [testnode : Work around broken wget on wheezy.] *************************** 2019-11-25T09:11:19.901 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.935 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.062) 0:03:51.316 ******* 2019-11-25T09:11:19.961 INFO:teuthology.task.ansible.out: TASK [testnode : Stop collectl] ************************************************ 2019-11-25T09:11:19.963 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:19.998 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:19 +0000 (0:00:00.062) 0:03:51.378 ******* 2019-11-25T09:11:20.024 INFO:teuthology.task.ansible.out: TASK [testnode : Add PATH to the teuthology_user .bashrc.] ********************* 2019-11-25T09:11:20.025 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.065 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.067) 0:03:51.446 ******* 2019-11-25T09:11:20.091 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if we need to edit /etc/profile.] **************** 2019-11-25T09:11:20.092 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.126 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.061) 0:03:51.507 ******* 2019-11-25T09:11:20.134 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-11-25T09:11:20.135 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-11-25T09:11:20.173 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.046) 0:03:51.553 ******* 2019-11-25T09:11:20.179 INFO:teuthology.task.ansible.out: TASK [testnode : Update /etc/profile if needed.] ******************************* 2019-11-25T09:11:20.181 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.222 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.049) 0:03:51.603 ******* 2019-11-25T09:11:20.229 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-11-25T09:11:20.230 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item={u'block_used': 13037835, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286649, u'size_available': 930966114304, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286649, "block_size": 4096, "block_total": 240324484, "block_used": 13037835, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930966114304, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-11-25T09:11:20.275 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.052) 0:03:51.655 ******* 2019-11-25T09:11:20.281 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:11:20.282 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"nvme_card_present": false}, "changed": false} 2019-11-25T09:11:20.320 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.045) 0:03:51.701 ******* 2019-11-25T09:11:20.368 INFO:teuthology.task.ansible.out: TASK [testnode : Check for NVMe drive] ***************************************** skipping: [smithi153.front.sepia.ceph.com] => (item=sda) => {"ansible_loop_var": "item", "changed": false, "item": "sda", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.389 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=nvme0n1) => {"ansible_facts": {"nvme_card_present": true}, "ansible_loop_var": "item", "changed": false, "item": "nvme0n1"} 2019-11-25T09:11:20.390 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=dm-4) => {"ansible_loop_var": "item", "changed": false, "item": "dm-4", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.401 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=dm-2) => {"ansible_loop_var": "item", "changed": false, "item": "dm-2", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.412 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=dm-3) => {"ansible_loop_var": "item", "changed": false, "item": "dm-3", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.424 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=dm-0) => {"ansible_loop_var": "item", "changed": false, "item": "dm-0", "skip_reason": "Conditional result was False"} skipping: [smithi153.front.sepia.ceph.com] => (item=dm-1) => {"ansible_loop_var": "item", "changed": false, "item": "dm-1", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.467 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.147) 0:03:51.848 ******* 2019-11-25T09:11:20.588 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-11-25T09:11:20.588 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.013917", "end": "2019-11-25 09:11:20.553665", "rc": 0, "start": "2019-11-25 09:11:20.539748", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:20.626 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.158) 0:03:52.007 ******* 2019-11-25T09:11:20.631 INFO:teuthology.task.ansible.out: TASK [testnode : Check for teuthology-lock command] **************************** 2019-11-25T09:11:20.632 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com -> localhost] => {"changed": true, "cmd": "which teuthology-lock", "delta": "0:00:00.003685", "end": "2019-11-25 09:11:20.607248", "rc": 0, "start": "2019-11-25 09:11:20.603563", "stderr": "", "stderr_lines": [], "stdout": "/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock", "stdout_lines": ["/home/teuthworker/src/git.ceph.com_git_teuthology_master/virtualenv/bin/teuthology-lock"]} 2019-11-25T09:11:20.674 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.047) 0:03:52.054 ******* 2019-11-25T09:11:20.679 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-11-25T09:11:20.680 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={u'block_used': 13037835, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286649, u'size_available': 930966114304, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286649, "block_size": 4096, "block_total": 240324484, "block_used": 13037835, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930966114304, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.720 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.046) 0:03:52.101 ******* 2019-11-25T09:11:20.726 INFO:teuthology.task.ansible.out: TASK [testnode : Mark system down if NVMe card missing] ************************ 2019-11-25T09:11:20.728 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.768 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.047) 0:03:52.149 ******* 2019-11-25T09:11:20.774 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-11-25T09:11:20.812 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.044) 0:03:52.193 ******* 2019-11-25T09:11:20.818 INFO:teuthology.task.ansible.out: TASK [testnode : Update description in paddles lock DB if NVMe card missing] *** 2019-11-25T09:11:20.827 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.864 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.051) 0:03:52.245 ******* 2019-11-25T09:11:20.871 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-11-25T09:11:20.872 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5L4BN', u'wwn-0x5000c500918e822f'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5L4BN', u'wwn': u'0x5000c500918e822f', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5L4BN-part1', u'wwn-0x5000c500918e822f-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L4BN", "wwn-0x5000c500918e822f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L4BN-part1", "wwn-0x5000c500918e822f-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L4BN", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-11-25T09:11:20.874 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500918e822f"}}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.895 INFO:teuthology.task.ansible.out: TASK [testnode : Fail rest of playbook due to missing NVMe card] *************** 2019-11-25T09:11:20.897 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.933 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.068) 0:03:52.314 ******* 2019-11-25T09:11:20.960 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure apt dependencies are installed] ********************* 2019-11-25T09:11:20.961 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:20.996 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:20 +0000 (0:00:00.062) 0:03:52.377 ******* 2019-11-25T09:11:21.469 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure rpm dependencies are installed] ********************* 2019-11-25T09:11:21.473 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["gdisk-0.8.10-2.el7.x86_64 providing gdisk is already installed", "7:device-mapper-1.02.149-8.el7.x86_64 providing device-mapper is already installed"]} 2019-11-25T09:11:21.511 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:21 +0000 (0:00:00.515) 0:03:52.892 ******* 2019-11-25T09:11:21.561 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-11-25T09:11:21.562 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item={u'block_used': 13037891, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286593, u'size_available': 930965884928, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_facts": {"root_disk": "/dev/sda"}, "ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286593, "block_size": 4096, "block_total": 240324484, "block_used": 13037891, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930965884928, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}} 2019-11-25T09:11:21.602 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:21 +0000 (0:00:00.091) 0:03:52.983 ******* 2019-11-25T09:11:21.905 INFO:teuthology.task.ansible.out: TASK [testnode : Compile list of non-root partitions] ************************** 2019-11-25T09:11:21.906 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": "lsblk --list --noheadings | grep part | grep -v sda | awk '{ print $1 }'", "delta": "0:00:00.013018", "end": "2019-11-25 09:11:21.872452", "rc": 0, "start": "2019-11-25 09:11:21.859434", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:21.945 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:21 +0000 (0:00:00.342) 0:03:53.326 ******* 2019-11-25T09:11:21.996 INFO:teuthology.task.ansible.out: TASK [testnode : Unmount any non-root mountpoints] ***************************** 2019-11-25T09:11:21.997 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'block_used': 13037891, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286593, u'size_available': 930965884928, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286593, "block_size": 4096, "block_total": 240324484, "block_used": 13037891, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930965884928, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:22.035 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:22 +0000 (0:00:00.090) 0:03:53.416 ******* 2019-11-25T09:11:22.062 INFO:teuthology.task.ansible.out: TASK [testnode : Wipe filesystems on non-root partitions] ********************** 2019-11-25T09:11:22.101 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:22 +0000 (0:00:00.066) 0:03:53.482 ******* 2019-11-25T09:11:22.150 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** 2019-11-25T09:11:22.151 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XFQY', u'wwn-0x5000c5009292246e'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XFQY', u'wwn': u'0x5000c5009292246e', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XFQY-part1', u'wwn-0x5000c5009292246e-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFQY", "wwn-0x5000c5009292246e"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFQY-part1", "wwn-0x5000c5009292246e-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XFQY", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-11-25T09:11:22.153 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c5009292246e"}}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:22.754 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-C2Ip6E-BjFk-fPTj-o31I-0rhC-IbVx-4F5rDT', u'nvme-INTEL_SSDPEDMD400G4_PHFT620400N7400BGN', u'nvme-nvme.8086-504846543632303430304e3734303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.588914", "end": "2019-11-25 09:11:22.712366", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-C2Ip6E-BjFk-fPTj-o31I-0rhC-IbVx-4F5rDT", "nvme-INTEL_SSDPEDMD400G4_PHFT620400N7400BGN", "nvme-nvme.8086-504846543632303430304e3734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:21.123452", "stderr": "", "stderr_lines": [], "stdout": "Creating new GP 2019-11-25T09:11:22.755 INFO:teuthology.task.ansible.out:T entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:23.984 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-cClGWb-Mj9Y-l8Mf-bR6z-LHqG-wKpL-7ptYMf', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300MV400BGN', u'nvme-nvme.8086-435646543632333330304d5634303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/nvme0n1 || sgdisk --zap-all /dev/nvme0n1", "delta": "0:00:01.516431", "end": "2019-11-25 09:11:23.946951", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-cClGWb-Mj9Y-l8Mf-bR6z-LHqG-wKpL-7ptYMf", "nvme-INTEL_SSDPEDMD400G4_CVFT623300MV400BGN", "nvme-nvme.8086-435646543632333330304d5634303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:22.430520", "stderr": "", "stderr_lines": [], "stdout": "Creating new GP 2019-11-25T09:11:23.986 INFO:teuthology.task.ansible.out:T entries.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:24.122 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx19091RV4du54FG0wQ2rFzySWgfpNZ77uq9R'], u'uuids': [u'28ba2955-8a63-43d5-b2cc-3a9c4c4895aa']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.067255", "end": "2019-11-25 09:11:24.078841", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx19091RV4du54FG0wQ2rFzySWgfpNZ77uq9R"], "labels": [], "masters": [], "uuids": ["28ba2955-8a63-43d5-b2cc-3a9c4c4895aa"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:23.011586", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:25.370 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4VpoH97QTkimMvvqZK2EQvBSAYpe3w9ee'], u'uuids': [u'eb5743b8-428c-42f1-b5c0-3afe107638fb']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-4 || sgdisk --zap-all /dev/dm-4", "delta": "0:00:01.060248", "end": "2019-11-25 09:11:25.328095", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4VpoH97QTkimMvvqZK2EQvBSAYpe3w9ee"], "labels": [], "masters": [], "uuids": ["eb5743b8-428c-42f1-b5c0-3afe107638fb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:24.267847", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:25.484 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190DquGBdAEN41IshOUYyDBxTWXFK21L1OE'], u'uuids': [u'8c98f0eb-6e17-43cb-9130-00573ca7a889']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.057012", "end": "2019-11-25 09:11:25.436160", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190DquGBdAEN41IshOUYyDBxTWXFK21L1OE"], "labels": [], "masters": [], "uuids": ["8c98f0eb-6e17-43cb-9130-00573ca7a889"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:24.379148", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:26.718 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP42we6lBudfsI01oMqsE9PQs2L1rwhCfwy'], u'uuids': [u'b9c1150f-86b4-49f9-b5a2-f49c689daf85']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-2 || sgdisk --zap-all /dev/dm-2", "delta": "0:00:01.050117", "end": "2019-11-25 09:11:26.679234", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP42we6lBudfsI01oMqsE9PQs2L1rwhCfwy"], "labels": [], "masters": [], "uuids": ["b9c1150f-86b4-49f9-b5a2-f49c689daf85"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:25.629117", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:26.864 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190ldXKb79foyXxVENqXN7ta0U9JdmGSPpf'], u'uuids': [u'34e5e083-4bc8-42d1-8fea-88af26089500']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.070983", "end": "2019-11-25 09:11:26.812306", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190ldXKb79foyXxVENqXN7ta0U9JdmGSPpf"], "labels": [], "masters": [], "uuids": ["34e5e083-4bc8-42d1-8fea-88af26089500"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:25.741323", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:28.077 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4LgEDjyLRGVohp8bIeeHALNfch4lsNscu'], u'uuids': [u'cd99cfd1-0cf6-4877-b7f7-5e9e55acd7a3']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-3 || sgdisk --zap-all /dev/dm-3", "delta": "0:00:01.056495", "end": "2019-11-25 09:11:28.036700", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4LgEDjyLRGVohp8bIeeHALNfch4lsNscu"], "labels": [], "masters": [], "uuids": ["cd99cfd1-0cf6-4877-b7f7-5e9e55acd7a3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:26.980205", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:28.225 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190ZrEeKPMljiQ3gYDftQeD2QYZGkDhfNE3'], u'uuids': [u'c6c8a666-0612-4f5c-9235-844a86bf0474']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.061135", "end": "2019-11-25 09:11:28.187645", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190ZrEeKPMljiQ3gYDftQeD2QYZGkDhfNE3"], "labels": [], "masters": [], "uuids": ["c6c8a666-0612-4f5c-9235-844a86bf0474"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:27.126510", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:29.455 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4QxY3a15dJx4cJaog9HcmW3CZmPhf9mIw'], u'uuids': [u'240ca3fc-3d71-406d-a0fd-21c85fd239ea']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-0 || sgdisk --zap-all /dev/dm-0", "delta": "0:00:01.066497", "end": "2019-11-25 09:11:29.413334", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4QxY3a15dJx4cJaog9HcmW3CZmPhf9mIw"], "labels": [], "masters": [], "uuids": ["240ca3fc-3d71-406d-a0fd-21c85fd239ea"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:28.346837", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:29.606 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190DkfuP4s6B35tPbpSSVo0lV0AzF75bgu3'], u'uuids': [u'38ab7911-589d-4ef9-abe9-3d7012b0f1b4']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.060056", "end": "2019-11-25 09:11:29.563108", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190DkfuP4s6B35tPbpSSVo0lV0AzF75bgu3"], "labels": [], "masters": [], "uuids": ["38ab7911-589d-4ef9-abe9-3d7012b0f1b4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:28.503052", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:29.649 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:29 +0000 (0:00:07.547) 0:04:01.030 ******* 2019-11-25T09:11:29.689 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-11-25T09:11:29.690 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5L4BN', u'wwn-0x5000c500918e822f'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5L4BN', u'wwn': u'0x5000c500918e822f', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5L4BN-part1', u'wwn-0x5000c500918e822f-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L4BN", "wwn-0x5000c500918e822f"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5L4BN-part1", "wwn-0x5000c500918e822f-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5L4BN", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-11-25T09:11:29.692 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c500918e822f"}}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:30.047 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-C2Ip6E-BjFk-fPTj-o31I-0rhC-IbVx-4F5rDT', u'nvme-INTEL_SSDPEDMD400G4_PHFT620400N7400BGN', u'nvme-nvme.8086-504846543632303430304e3734303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.061221", "end": "2019-11-25 09:11:30.015604", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-C2Ip6E-BjFk-fPTj-o31I-0rhC-IbVx-4F5rDT", "nvme-INTEL_SSDPEDMD400G4_PHFT620400N7400BGN", "nvme-nvme.8086-504846543632303430304e3734303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:29.954383", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) co 2019-11-25T09:11:30.048 INFO:teuthology.task.ansible.out:pied, 0.0551523 s, 2.1 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0551523 s, 2.1 GB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:30.484 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx19091RV4du54FG0wQ2rFzySWgfpNZ77uq9R'], u'uuids': [u'28ba2955-8a63-43d5-b2cc-3a9c4c4895aa']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.180065", "end": "2019-11-25 09:11:30.445377", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx19091RV4du54FG0wQ2rFzySWgfpNZ77uq9R"], "labels": [], "masters": [], "uuids": ["28ba2955-8a63-43d5-b2cc-3a9c4c4895aa"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:30.265312", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.175417 s, 658 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.175417 s, 658 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:30.828 INFO:teuthology.task.ansible.out: TASK [testnode : Zap all non-root disks] *************************************** changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4hnuKBEAgZc26aF8qdlhr2lExbbPapZMl'], u'uuids': [u'4c758906-3365-4204-a5b8-01d2d32a89ee']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "sgdisk --zap-all /dev/dm-1 || sgdisk --zap-all /dev/dm-1", "delta": "0:00:01.069132", "end": "2019-11-25 09:11:30.778649", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4hnuKBEAgZc26aF8qdlhr2lExbbPapZMl"], "labels": [], "masters": [], "uuids": ["4c758906-3365-4204-a5b8-01d2d32a89ee"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:29.709517", "stderr": "", "stderr_lines": [], "stdout": "Creating new GPT entries.\nWarning: The kernel is still using the old partition table.\nThe new table will be used at the next reboot.\nGPT data structures destroyed! You may now partition the disk using fdisk or\nother utilities.", "stdout_lines": ["Creating new GPT entries.", "Warning: The kernel is still using the old partition table.", "The new table will be used at the next reboot.", "GPT data structures destroyed! You may now partition the disk using fdisk or", "other utilities."]} 2019-11-25T09:11:30.875 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:30 +0000 (0:00:01.226) 0:04:02.256 ******* 2019-11-25T09:11:30.915 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-11-25T09:11:30.916 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': u'ATA', u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XFQY', u'wwn-0x5000c5009292246e'], u'uuids': []}, u'sas_device_handle': None, u'host': u'SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)', u'support_discard': u'0', u'serial': u'Z1W5XFQY', u'wwn': u'0x5000c5009292246e', u'holders': [], u'size': u'931.51 GB', u'scheduler_mode': u'deadline', u'rotational': u'1', u'sectors': u'1953525168', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'model': u'ST1000NM0033-9ZM', u'partitions': {u'sda1': {u'start': u'2048', u'sectorsize': 512, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'sectors': u'1953522688', u'holders': [], u'links': {u'masters': [], u'labels': [], u'ids': [u'ata-ST1000NM0033-9ZM173_Z1W5XFQY-part1', u'wwn-0x5000c5009292246e-part1'], u'uuids': [u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67']}, u'size': u'931.51 GB'}}}, 'key': u'sda'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "sda", "value": {"holders": [], "host": "SATA controller: Intel Corporation C610/X99 series chipset 6-Port SATA Controller [AHCI mode] (rev 05)", "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFQY", "wwn-0x5000c5009292246e"], "labels": [], "masters": [], "uuids": []}, "model": "ST1000NM0033-9ZM", "partitions": {"sda1": {"holders": [], "links": {"ids": ["ata-ST1000NM0033-9ZM173_Z1W5XFQY-part1", "wwn-0x5000c5009292246e-part1"], "labels": [], "masters": [], "uuids": ["60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"]}, "sectors": "1953522688", "sectorsize": 512, "size": "931.51 GB", "start": "2048", "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}}, "removable": "0", "rotational": "1", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "deadline", "sectors": "1953525168", "sectorsize": "512", "serial": "Z1W5XFQY", "size": "931.51 GB", "support_discard": "0", "vendor": " 2019-11-25T09:11:30.918 INFO:teuthology.task.ansible.out:ATA", "virtual": 1, "wwn": "0x5000c5009292246e"}}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:30.980 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190DquGBdAEN41IshOUYyDBxTWXFK21L1OE'], u'uuids': [u'8c98f0eb-6e17-43cb-9130-00573ca7a889']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.198538", "end": "2019-11-25 09:11:30.935167", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190DquGBdAEN41IshOUYyDBxTWXFK21L1OE"], "labels": [], "masters": [], "uuids": ["8c98f0eb-6e17-43cb-9130-00573 2019-11-25T09:11:30.982 INFO:teuthology.task.ansible.out:ca7a889"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:30.736629", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.192083 s, 600 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.192083 s, 600 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:31.308 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [u'dm-0', u'dm-1', u'dm-2', u'dm-3', u'dm-4'], u'labels': [], u'ids': [u'lvm-pv-uuid-cClGWb-Mj9Y-l8Mf-bR6z-LHqG-wKpL-7ptYMf', u'nvme-INTEL_SSDPEDMD400G4_CVFT623300MV400BGN', u'nvme-nvme.8086-435646543632333330304d5634303042474e-494e54454c205353445045444d443430304734-00000001'], u'uuids': []}, u'sas_device_handle': None, u'host': u'Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)', u'support_discard': u'512', u'model': u'INTEL SSDPEDMD400G4', u'size': u'372.61 GB', u'scheduler_mode': u'none', u'rotational': u'0', u'sectors': u'781422768', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [u'vg_nvme-lv_5', u'vg_nvme-lv_4', u'vg_nvme-lv_3', u'vg_nvme-lv_2', u'vg_nvme-lv_1'], u'partitions': {}}, 'key': u'nvme0n1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/nvme0n1 bs=1M count=110", "delta": "0:00:00.088202", "end": "2019-11-25 09:11:31.272090", "item": {"key": "nvme0n1", "value": {"holders": ["vg_nvme-lv_5", "vg_nvme-lv_4", "vg_nvme-lv_3", "vg_nvme-lv_2", "vg_nvme-lv_1"], "host": "Non-Volatile memory controller: Intel Corporation PCIe Data Center SSD (rev 01)", "links": {"ids": ["lvm-pv-uuid-cClGWb-Mj9Y-l8Mf-bR6z-LHqG-wKpL-7ptYMf", "nvme-INTEL_SSDPEDMD400G4_CVFT623300MV400BGN", "nvme-nvme.8086-435646543632333330304d5634303042474e-494e54454c205353445045444d443430304734-00000001"], "labels": [], "masters": ["dm-0", "dm-1", "dm-2", "dm-3", "dm-4"], "uuids": []}, "model": "INTEL SSDPEDMD400G4", "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "none", "sectors": "781422768", "sectorsize": "512", "size": "372.61 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:31.183888", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) co 2019-11-25T09:11:31.309 INFO:teuthology.task.ansible.out:pied, 0.0817656 s, 1.4 GB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.0817656 s, 1.4 GB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:31.479 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190ldXKb79foyXxVENqXN7ta0U9JdmGSPpf'], u'uuids': [u'34e5e083-4bc8-42d1-8fea-88af26089500']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.202887", "end": "2019-11-25 09:11:31.434198", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190ldXKb79foyXxVENqXN7ta0U9JdmGSPpf"], "labels": [], "masters": [], "uuids": ["34e5e083-4bc8-42d1-8fea-88af26089500"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:31.231311", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.196845 s, 586 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.196845 s, 586 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:31.740 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_1', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4VpoH97QTkimMvvqZK2EQvBSAYpe3w9ee'], u'uuids': [u'eb5743b8-428c-42f1-b5c0-3afe107638fb']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-4'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-4 bs=1M count=110", "delta": "0:00:00.177223", "end": "2019-11-25 09:11:31.708839", "item": {"key": "dm-4", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_1", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4VpoH97QTkimMvvqZK2EQvBSAYpe3w9ee"], "labels": [], "masters": [], "uuids": ["eb5743b8-428c-42f1-b5c0-3afe107638fb"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:31.531616", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.170664 s, 676 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.170664 s, 676 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:31.963 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190ZrEeKPMljiQ3gYDftQeD2QYZGkDhfNE3'], u'uuids': [u'c6c8a666-0612-4f5c-9235-844a86bf0474']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.180439", "end": "2019-11-25 09:11:31.928754", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190ZrEeKPMljiQ3gYDftQeD2QYZGkDhfNE3"], "labels": [], "masters": [], "uuids": ["c6c8a666-0612-4f5c-9235-844a86bf0474"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:31.748315", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.173903 s, 663 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.173903 s, 663 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:32.274 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_3', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP42we6lBudfsI01oMqsE9PQs2L1rwhCfwy'], u'uuids': [u'b9c1150f-86b4-49f9-b5a2-f49c689daf85']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-2'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-2 bs=1M count=110", "delta": "0:00:00.204632", "end": "2019-11-25 09:11:32.231496", "item": {"key": "dm-2", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_3", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP42we6lBudfsI01oMqsE9PQs2L1rwhCfwy"], "labels": [], "masters": [], "uuids": ["b9c1150f-86b4-49f9-b5a2-f49c689daf85"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:32.026864", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.198365 s, 581 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.198365 s, 581 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:32.452 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190DkfuP4s6B35tPbpSSVo0lV0AzF75bgu3'], u'uuids': [u'38ab7911-589d-4ef9-abe9-3d7012b0f1b4']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.169803", "end": "2019-11-25 09:11:32.417974", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-rvA0c4bMbvomuYoleUMn84BAs6obx190DkfuP4s6B35tPbpSSVo0lV0AzF75bgu3"], "labels": [], "masters": [], "uuids": ["38ab7911-589d-4ef9-abe9-3d7012b0f1b4"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:32.248171", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.163227 s, 707 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.163227 s, 707 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:32.506 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:32 +0000 (0:00:01.630) 0:04:03.887 ******* 2019-11-25T09:11:32.792 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_2', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4LgEDjyLRGVohp8bIeeHALNfch4lsNscu'], u'uuids': [u'cd99cfd1-0cf6-4877-b7f7-5e9e55acd7a3']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-3'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-3 bs=1M count=110", "delta": "0:00:00.202131", "end": "2019-11-25 09:11:32.749898", "item": {"key": "dm-3", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_2", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4LgEDjyLRGVohp8bIeeHALNfch4lsNscu"], "labels": [], "masters": [], "uuids": ["cd99cfd1-0cf6-4877-b7f7-5e9e55acd7a3"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:32.547767", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.195917 s, 589 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.195917 s, 589 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:32.929 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-11-25T09:11:32.940 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.090910", "end": "2019-11-25 09:11:32.888941", "rc": 0, "start": "2019-11-25 09:11:32.798031", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:32.983 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:32 +0000 (0:00:00.476) 0:04:04.363 ******* 2019-11-25T09:11:33.311 INFO:teuthology.task.ansible.out: TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_5', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4QxY3a15dJx4cJaog9HcmW3CZmPhf9mIw'], u'uuids': [u'240ca3fc-3d71-406d-a0fd-21c85fd239ea']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'14.90 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'31252480', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-0'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-0 bs=1M count=110", "delta": "0:00:00.204605", "end": "2019-11-25 09:11:33.261669", "item": {"key": "dm-0", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_5", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4QxY3a15dJx4cJaog9HcmW3CZmPhf9mIw"], "labels": [], "masters": [], "uuids": ["240ca3fc-3d71-406d-a0fd-21c85fd239ea"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "31252480", "sectorsize": "512", "size": "14.90 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:33.057064", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.198293 s, 582 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.198293 s, 582 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:33.353 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-11-25T09:11:33.355 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.048033", "end": "2019-11-25 09:11:33.325891", "rc": 0, "start": "2019-11-25 09:11:33.277858", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:33.395 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:33 +0000 (0:00:00.412) 0:04:04.776 ******* 2019-11-25T09:11:33.449 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-11-25T09:11:33.511 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:33 +0000 (0:00:00.115) 0:04:04.891 ******* 2019-11-25T09:11:33.537 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* 2019-11-25T09:11:33.538 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:33.577 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:33 +0000 (0:00:00.066) 0:04:04.958 ******* 2019-11-25T09:11:33.580 INFO:teuthology.task.ansible.out: 2019-11-25T09:11:33.613 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** 2019-11-25T09:11:33.615 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:33.652 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:33 +0000 (0:00:00.074) 0:04:05.033 ******* 2019-11-25T09:11:33.680 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-11-25T09:11:33.682 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:33.728 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:33 +0000 (0:00:00.076) 0:04:05.109 ******* 2019-11-25T09:11:33.771 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-11-25T09:11:33.773 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : Blow away lingering OSD data and FSIDs] *********************** 2019-11-25T09:11:33.775 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'sectorsize': u'512', u'vendor': None, u'links': {u'masters': [], u'labels': [], u'ids': [u'dm-name-vg_nvme-lv_4', u'dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4hnuKBEAgZc26aF8qdlhr2lExbbPapZMl'], u'uuids': [u'4c758906-3365-4204-a5b8-01d2d32a89ee']}, u'sas_device_handle': None, u'host': u'', u'support_discard': u'512', u'model': None, u'size': u'89.40 GB', u'scheduler_mode': u'', u'rotational': u'0', u'sectors': u'187490304', u'sas_address': None, u'virtual': 1, u'removable': u'0', u'holders': [], u'partitions': {}}, 'key': u'dm-1'}) => {"ansible_loop_var": "item", "changed": true, "cmd": "dd if=/dev/zero of=/dev/dm-1 bs=1M count=110", "delta": "0:00:00.175425", "end": "2019-11-25 09:11:33.733679", "item": {"key": "dm-1", "value": {"holders": [], "host": "", "links": {"ids": ["dm-name-vg_nvme-lv_4", "dm-uuid-LVM-R0KTZXq3eUBr2w0lUIuYRw9f1WlkelP4hnuKBEAgZc26aF8qdlhr2lExbbPapZMl"], "labels": [], "masters": [], "uuids": ["4c758906-3365-4204-a5b8-01d2d32a89ee"]}, "model": null, "partitions": {}, "removable": "0", "rotational": "0", "sas_address": null, "sas_device_handle": null, "scheduler_mode": "", "sectors": "187490304", "sectorsize": "512", "size": "89.40 GB", "support_discard": "512", "vendor": null, "virtual": 1}}, "rc": 0, "start": "2019-11-25 09:11:33.558254", "stderr": "110+0 records in\n110+0 records out\n115343360 bytes (115 MB) copied, 0.169885 s, 679 MB/s", "stderr_lines": ["110+0 records in", "110+0 records out", "115343360 bytes (115 MB) copied, 0.169885 s, 679 MB/s"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:33.821 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:33 +0000 (0:00:00.092) 0:04:05.202 ******* 2019-11-25T09:11:33.858 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:33 +0000 (0:00:00.037) 0:04:05.239 ******* 2019-11-25T09:11:34.141 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-11-25T09:11:34.142 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:11:34.202 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.343) 0:04:05.583 ******* 2019-11-25T09:11:34.254 INFO:teuthology.task.ansible.out: TASK [testnode : Remove all LVM data] ****************************************** 2019-11-25T09:11:34.255 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"attempts": 1, "changed": true, "cmd": "dmsetup remove_all --force", "delta": "0:00:00.082798", "end": "2019-11-25 09:11:34.213001", "rc": 0, "start": "2019-11-25 09:11:34.130203", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:34.257 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ skipping: [smithi081.front.sepia.ceph.com] => (item={u'block_used': 13037835, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286649, u'size_available': 930966114304, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286649, "block_size": 4096, "block_total": 240324484, "block_used": 13037835, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930966114304, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:34.308 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.105) 0:04:05.689 ******* 2019-11-25T09:11:34.348 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.039) 0:04:05.729 ******* 2019-11-25T09:11:34.354 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-11-25T09:11:34.354 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:34.393 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.045) 0:04:05.774 ******* 2019-11-25T09:11:34.431 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:11:34.432 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:34.472 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.078) 0:04:05.853 ******* 2019-11-25T09:11:34.502 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:11:34.504 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:34.551 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.071) 0:04:05.924 ******* 2019-11-25T09:11:34.580 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-11-25T09:11:34.582 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:34.620 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.077) 0:04:06.001 ******* 2019-11-25T09:11:34.648 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-11-25T09:11:34.650 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:34.696 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.067) 0:04:06.069 ******* 2019-11-25T09:11:34.715 INFO:teuthology.task.ansible.out: TASK [testnode : Check for physical volumes] *********************************** 2019-11-25T09:11:34.716 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "cmd": "pvdisplay | grep 'PV Name' | awk '{ print $3 }'", "delta": "0:00:00.049690", "end": "2019-11-25 09:11:34.678686", "rc": 0, "start": "2019-11-25 09:11:34.628996", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} TASK [testnode : Create logical_volumes dict] ********************************** 2019-11-25T09:11:34.718 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:34.757 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.068) 0:04:06.138 ******* 2019-11-25T09:11:34.792 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.034) 0:04:06.173 ******* 2019-11-25T09:11:34.820 INFO:teuthology.task.ansible.out: TASK [testnode : Remove physical volumes] ************************************** 2019-11-25T09:11:34.863 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.070) 0:04:06.244 ******* 2019-11-25T09:11:34.915 INFO:teuthology.task.ansible.out: TASK [testnode : Write a new partition table to {{ item.value.device }}] ******* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:34.955 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:34 +0000 (0:00:00.092) 0:04:06.336 ******* 2019-11-25T09:11:34.987 INFO:teuthology.task.ansible.out: TASK [testnode : Write partition entries to {{ item.0.device }}] *************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:35.035 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.075) 0:04:06.411 ******* 2019-11-25T09:11:35.059 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-11-25T09:11:35.065 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:35.106 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.076) 0:04:06.487 ******* 2019-11-25T09:11:35.137 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs for {{ item.0.device }}] ****************** 2019-11-25T09:11:35.139 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:35.176 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.069) 0:04:06.557 ******* 2019-11-25T09:11:35.496 INFO:teuthology.task.ansible.out: TASK [testnode : Edit lvm.conf to support LVM on kRBD.] ************************ 2019-11-25T09:11:35.498 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"backup": "", "changed": false, "msg": ""} 2019-11-25T09:11:35.539 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.362) 0:04:06.920 ******* 2019-11-25T09:11:35.588 INFO:teuthology.task.ansible.out: TASK [testnode : Set root disk] ************************************************ 2019-11-25T09:11:35.590 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={u'block_used': 13037891, u'uuid': u'60ab4ba9-cdc2-4668-aaad-a5769fd8ef67', u'size_total': 984369086464, u'block_total': 240324484, u'mount': u'/', u'block_available': 227286593, u'size_available': 930965884928, u'fstype': u'ext4', u'inode_total': 61054976, u'inode_available': 60966954, u'device': u'/dev/sda1', u'inode_used': 88022, u'block_size': 4096, u'options': u'rw,seclabel,relatime,data=ordered'}) => {"ansible_loop_var": "item", "changed": false, "item": {"block_available": 227286593, "block_size": 4096, "block_total": 240324484, "block_used": 13037891, "device": "/dev/sda1", "fstype": "ext4", "inode_available": 60966954, "inode_total": 61054976, "inode_used": 88022, "mount": "/", "options": "rw,seclabel,relatime,data=ordered", "size_available": 930965884928, "size_total": 984369086464, "uuid": "60ab4ba9-cdc2-4668-aaad-a5769fd8ef67"}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:35.641 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.101) 0:04:07.021 ******* 2019-11-25T09:11:35.646 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-11-25T09:11:35.647 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-11-25T09:11:35.690 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.049) 0:04:07.071 ******* 2019-11-25T09:11:35.696 INFO:teuthology.task.ansible.out: TASK [testnode : Combine list of non-root disks] ******************************* 2019-11-25T09:11:35.697 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:35.756 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.066) 0:04:07.137 ******* 2019-11-25T09:11:35.804 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:11:35.806 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:35.841 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.084) 0:04:07.222 ******* 2019-11-25T09:11:35.868 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:11:35.869 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:35.924 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.083) 0:04:07.305 ******* 2019-11-25T09:11:35.954 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume_groups dict] ************************************ 2019-11-25T09:11:35.956 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:35.990 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:35 +0000 (0:00:00.065) 0:04:07.371 ******* 2019-11-25T09:11:36.026 INFO:teuthology.task.ansible.out: TASK [testnode : Determine desired logical volume percentage size] ************* 2019-11-25T09:11:36.028 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:36.064 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:36 +0000 (0:00:00.073) 0:04:07.445 ******* 2019-11-25T09:11:36.106 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical_volumes dict] ********************************** 2019-11-25T09:11:36.108 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:36.147 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:36 +0000 (0:00:00.083) 0:04:07.528 ******* 2019-11-25T09:11:36.432 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-11-25T09:11:36.433 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:36.683 INFO:teuthology.task.ansible.out: TASK [testnode : Create volume group(s)] *************************************** 2019-11-25T09:11:36.684 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'pvs': u'/dev/nvme0n1'}, 'key': u'vg_nvme'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "vg_nvme", "value": {"pvs": "/dev/nvme0n1"}}} 2019-11-25T09:11:36.722 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:36 +0000 (0:00:00.575) 0:04:08.103 ******* 2019-11-25T09:11:36.904 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* 2019-11-25T09:11:36.905 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:37.219 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:37.365 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:37.676 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:37.834 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:38.176 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:38.311 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:38.351 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:38 +0000 (0:00:01.628) 0:04:09.731 ******* 2019-11-25T09:11:38.639 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:38.660 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-11-25T09:11:38.662 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-11-25T09:11:38.700 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:38 +0000 (0:00:00.349) 0:04:10.081 ******* 2019-11-25T09:11:38.740 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-11-25T09:11:38.742 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:39.039 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-25T09:11:39.168 INFO:teuthology.task.ansible.out: TASK [testnode : Create logical volume(s)] ************************************* changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": ""} 2019-11-25T09:11:39.208 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:39 +0000 (0:00:00.507) 0:04:10.589 ******* 2019-11-25T09:11:39.353 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-11-25T09:11:39.355 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-25T09:11:39.545 INFO:teuthology.task.ansible.out: TASK [testnode : Erase /scratch_devs so we know it's accurate] ***************** 2019-11-25T09:11:39.546 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "path": "/scratch_devs", "state": "absent"} 2019-11-25T09:11:39.589 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:39 +0000 (0:00:00.381) 0:04:10.970 ******* 2019-11-25T09:11:39.639 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-11-25T09:11:39.641 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={'value': {u'vg': u'vg_nvme', u'size': u'14.9G'}, 'key': u'lv_5'}) => {"ansible_loop_var": "item", "changed": false, "item": {"key": "lv_5", "value": {"size": "14.9G", "vg": "vg_nvme"}}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:39.677 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-25T09:11:39.927 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_4'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_4", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-25T09:11:39.964 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-25T09:11:40.021 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:40 +0000 (0:00:00.428) 0:04:11.399 ******* 2019-11-25T09:11:40.249 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_3'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_3", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-25T09:11:40.344 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-11-25T09:11:40.345 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-25T09:11:40.405 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:40 +0000 (0:00:00.384) 0:04:11.784 ******* 2019-11-25T09:11:40.442 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-11-25T09:11:40.444 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-11-25T09:11:40.501 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:40 +0000 (0:00:00.098) 0:04:11.882 ******* 2019-11-25T09:11:40.573 INFO:teuthology.task.ansible.out: TASK [testnode : Write /scratch_devs] ****************************************** 2019-11-25T09:11:40.574 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_2'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_2", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-25T09:11:40.875 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => (item={'value': {u'scratch_dev': True, u'vg': u'vg_nvme', u'size': u'89.4G'}, 'key': u'lv_1'}) => {"ansible_loop_var": "item", "backup": "", "changed": true, "item": {"key": "lv_1", "value": {"scratch_dev": true, "size": "89.4G", "vg": "vg_nvme"}}, "msg": "line added"} 2019-11-25T09:11:40.930 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:40 +0000 (0:00:00.428) 0:04:12.311 ******* 2019-11-25T09:11:41.217 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-11-25T09:11:41.219 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true} 2019-11-25T09:11:41.258 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:41 +0000 (0:00:00.328) 0:04:12.639 ******* 2019-11-25T09:11:41.264 INFO:teuthology.task.ansible.out: TASK [testnode : Create /var/lib/ceph] ***************************************** 2019-11-25T09:11:41.265 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "gid": 0, "group": "root", "mode": "0755", "owner": "root", "path": "/var/lib/ceph", "secontext": "unconfined_u:object_r:var_lib_t:s0", "size": 4096, "state": "directory", "uid": 0} 2019-11-25T09:11:41.313 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:41 +0000 (0:00:00.054) 0:04:12.693 ******* 2019-11-25T09:11:41.349 INFO:teuthology.task.ansible.out: TASK [testnode : Set xfs_opts on newer OSes] *********************************** 2019-11-25T09:11:41.351 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"xfs_opts": "-m crc=0,finobt=0"}, "changed": false} 2019-11-25T09:11:41.413 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:41 +0000 (0:00:00.100) 0:04:12.794 ******* 2019-11-25T09:11:41.852 INFO:teuthology.task.ansible.out: TASK [testnode : Create xfs filesystem on /dev/vg_nvme/lv_5] ******************* 2019-11-25T09:11:41.854 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true} 2019-11-25T09:11:41.892 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:41 +0000 (0:00:00.478) 0:04:13.273 ******* 2019-11-25T09:11:42.013 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-11-25T09:11:42.051 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.158) 0:04:13.431 ******* 2019-11-25T09:11:42.080 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.116 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.064) 0:04:13.496 ******* 2019-11-25T09:11:42.142 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-11-25T09:11:42.144 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.179 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.063) 0:04:13.560 ******* 2019-11-25T09:11:42.205 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-11-25T09:11:42.207 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.242 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.063) 0:04:13.623 ******* 2019-11-25T09:11:42.269 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-11-25T09:11:42.271 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.306 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.063) 0:04:13.687 ******* 2019-11-25T09:11:42.312 INFO:teuthology.task.ansible.out: TASK [testnode : Mount /dev/vg_nvme/lv_5 to /var/lib/ceph] ********************* 2019-11-25T09:11:42.313 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "dump": "0", "fstab": "/etc/fstab", "fstype": "xfs", "name": "/var/lib/ceph", "opts": "defaults,nofail", "passno": "0", "src": "/dev/vg_nvme/lv_5"} 2019-11-25T09:11:42.351 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.044) 0:04:13.732 ******* 2019-11-25T09:11:42.356 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-11-25T09:11:42.357 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.394 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.043) 0:04:13.775 ******* 2019-11-25T09:11:42.401 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the teuthology user.] *********** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.440 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.045) 0:04:13.821 ******* 2019-11-25T09:11:42.446 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* 2019-11-25T09:11:42.448 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.485 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.044) 0:04:13.866 ******* 2019-11-25T09:11:42.500 INFO:teuthology.task.ansible.out: TASK [testnode : Add CPAN config directory for the root user.] ***************** 2019-11-25T09:11:42.502 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.536 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.051) 0:04:13.917 ******* 2019-11-25T09:11:42.542 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-11-25T09:11:42.543 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.582 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.046) 0:04:13.963 ******* 2019-11-25T09:11:42.588 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for the teuthology user.] ****************** 2019-11-25T09:11:42.589 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.628 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.045) 0:04:14.009 ******* 2019-11-25T09:11:42.655 INFO:teuthology.task.ansible.out: TASK [testnode : Upload CPAN config for root.] ********************************* 2019-11-25T09:11:42.656 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.692 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.064) 0:04:14.073 ******* 2019-11-25T09:11:42.719 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure perl-doc is installed on apt systems.] ***************** 2019-11-25T09:11:42.720 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.756 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.063) 0:04:14.137 ******* 2019-11-25T09:11:42.796 INFO:teuthology.task.ansible.out: TASK [testnode : Check to see if Amazon::S3 is installed.] ********************* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.832 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.073) 0:04:14.210 ******* 2019-11-25T09:11:42.856 INFO:teuthology.task.ansible.out: TASK [testnode : Install Amazon::S3.] ****************************************** 2019-11-25T09:11:42.857 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:42.893 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:42 +0000 (0:00:00.063) 0:04:14.274 ******* 2019-11-25T09:11:43.070 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-11-25T09:11:43.124 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:43 +0000 (0:00:00.230) 0:04:14.504 ******* 2019-11-25T09:11:43.151 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** 2019-11-25T09:11:43.153 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:43.189 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:43 +0000 (0:00:00.065) 0:04:14.570 ******* 2019-11-25T09:11:43.226 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:43.264 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:43 +0000 (0:00:00.074) 0:04:14.645 ******* 2019-11-25T09:11:43.365 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on rpm based systems.] ******************** 2019-11-25T09:11:43.366 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["ntp-4.2.6p5-28.el7.centos.x86_64 providing ntp is already installed"]} 2019-11-25T09:11:43.406 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:43 +0000 (0:00:00.141) 0:04:14.786 ******* 2019-11-25T09:11:43.447 INFO:teuthology.task.ansible.out: TASK [testnode : Install ntp package on apt based systems.] ******************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:43.499 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:43 +0000 (0:00:00.093) 0:04:14.880 ******* 2019-11-25T09:11:43.536 INFO:teuthology.task.ansible.out: TASK [testnode : Remove racy /etc/network/if-up.d/ntpdate on xenial] *********** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:43.566 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:43 +0000 (0:00:00.066) 0:04:14.947 ******* 2019-11-25T09:11:44.206 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ 2019-11-25T09:11:44.208 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-11-25T09:11:44.248 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:44 +0000 (0:00:00.682) 0:04:15.629 ******* 2019-11-25T09:11:44.275 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-11-25T09:11:44.276 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:44.348 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:44 +0000 (0:00:00.099) 0:04:15.728 ******* 2019-11-25T09:11:44.470 INFO:teuthology.task.ansible.out: TASK [testnode : Create the ntp.conf file.] ************************************ ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "645fd5116197cd9da73b328bc11bf633a40d80e1", "dest": "/etc/ntp.conf", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "path": "/etc/ntp.conf", "secontext": "system_u:object_r:net_conf_t:s0", "size": 2767, "state": "file", "uid": 0} 2019-11-25T09:11:44.517 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:44 +0000 (0:00:00.164) 0:04:15.893 ******* 2019-11-25T09:11:44.539 INFO:teuthology.task.ansible.out: TASK [testnode : Create the chrony.conf file] ********************************** 2019-11-25T09:11:44.543 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:44.581 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:44 +0000 (0:00:00.068) 0:04:15.962 ******* 2019-11-25T09:11:44.893 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-11-25T09:11:44.895 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "-.mount sntp.service system.slice syslog.target ntpdate.service systemd-journald.socket basic.target tmp.mount", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "multi-user.target chronyd.service shutdown.target", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-11-25T09:11:44.896 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-11-25T09:11:44.898 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-11-25T09:11:44.936 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:44 +0000 (0:00:00.355) 0:04:16.317 ******* 2019-11-25T09:11:45.862 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-11-25T09:11:45.918 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:45 +0000 (0:00:00.981) 0:04:17.299 ******* 2019-11-25T09:11:45.945 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* 2019-11-25T09:11:45.946 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:45.982 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:45 +0000 (0:00:00.063) 0:04:17.363 ******* 2019-11-25T09:11:46.077 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ntpd is running.] *********************************** 2019-11-25T09:11:46.079 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "enabled": true, "name": "ntpd", "state": "started", "status": {"ActiveEnterTimestampMonotonic": "0", "ActiveExitTimestampMonotonic": "0", "ActiveState": "inactive", "After": "ntpdate.service basic.target tmp.mount syslog.target system.slice sntp.service -.mount systemd-journald.socket", "AllowIsolate": "no", "AmbientCapabilities": "0", "AssertResult": "no", "AssertTimestampMonotonic": "0", "Before": "shutdown.target multi-user.target chronyd.service", "BlockIOAccounting": "no", "BlockIOWeight": "18446744073709551615", "CPUAccounting": "no", "CPUQuotaPerSecUSec": "infinity", "CPUSchedulingPolicy": "0", "CPUSchedulingPriority": "0", "CPUSchedulingResetOnFork": "no", "CPUShares": "18446744073709551615", "CanIsolate": "no", "CanReload": "no", "CanStart": "yes", "CanStop": "yes", "CapabilityBoundingSet": "18446744073709551615", "ConditionResult": "no", "ConditionTimestampMonotonic": "0", "ConflictedBy": "chronyd.service", "Conflicts": "shutdown.target", "ControlPID": "0", "DefaultDependencies": "yes", "Delegate": "no", "Description": "Network Time Service", "DevicePolicy": "auto", "EnvironmentFile": "/etc/sysconfig/ntpd (ignore_errors=yes)", "ExecMainCode": "0", "ExecMainExitTimestampMonotonic": "0", "ExecMainPID": "0", "ExecMainStartTimestampMonotonic": "0", "ExecMainStatus": "0", "ExecStart": "{ path=/usr/sbin/ntpd ; argv[]=/usr/sbin/ntpd -u ntp:ntp $OPTIONS ; ignore_errors=no ; start_time=[n/a] ; stop_time=[n/a] ; pid=0 ; code=(null) ; status=0/0 }", "FailureAction": "none", "FileDescriptorStoreMax": "0", "FragmentPath": "/usr/lib/systemd/system/ntpd.service", "GuessMainPID": "yes", "IOScheduling": "0", "Id": "ntpd.service", "IgnoreOnIsolate": "no", "IgnoreOnSnapshot": "no", "IgnoreSIGPIPE": "yes", "InactiveEnterTimestampMonotonic": "0", "InactiveExitTimestampMonotonic": "0", "JobTimeoutAction": "none", "JobTimeoutUSec": "0", "KillMode": "control-group", "KillSignal": "15", "LimitAS": "1844674407370955161 2019-11-25T09:11:46.081 INFO:teuthology.task.ansible.out:5", "LimitCORE": "18446744073709551615", "LimitCPU": "18446744073709551615", "LimitDATA": "18446744073709551615", "LimitFSIZE": "18446744073709551615", "LimitLOCKS": "18446744073709551615", "LimitMEMLOCK": "65536", "LimitMSGQUEUE": "819200", "LimitNICE": "0", "LimitNOFILE": "4096", "LimitNPROC": "127511", "LimitRSS": "18446744073709551615", "LimitRTPRIO": "0", "LimitRTTIME": "18446744073709551615", "LimitSIGPENDING": "127511", "LimitSTACK": "18446744073709551615", "LoadState": "loaded", "MainPID": "0", "MemoryAccounting": "no", "MemoryCurrent": "18446744073709551615", "MemoryLimit": "18446744073709551615", "MountFlags": "0", "Names": "ntpd.service", "NeedDaemonReload": "no", "Nice": "0", "NoNewPrivileges": "no", "NonBlocking": "no", "NotifyAccess": "none", "OOMScoreAdjust": "0", "OnFailureJobMode": "replace", "PermissionsStartOnly": "no", "PrivateDevices": "no", "PrivateNetwork": "no", "PrivateTmp": "yes", "ProtectHome": "no", "ProtectSystem": "no", "RefuseManualStart": "no", "RefuseManualStop": "no", "RemainAfterExit": "no", "Requires": "basic.target -.mount", "RequiresMountsFor": "/var/tmp", "Restart": "no", "RestartUSec": "100ms", "Result": "success", "RootDirectoryStartOnly": "no", "RuntimeDirectoryMode": "0755", "SameProcessGroup": "no", "SecureBits": "0", "SendSIGHUP": "no", "SendSIGKILL": "yes", "Slice": "system.slice", "StandardError": "inherit", "StandardInput": "null", "StandardOutput": "journal", "StartLimitAction": "none", "StartLimitBurst": "5", "StartLimitInterval": "10000000", "StartupBlockIOWeight": "18446744073709551615", "StartupCPUShares": "18446744073709551615", "StatusErrno": "0", "StopWhenUnneeded": "no", "SubState": "dead", "SyslogLevelPrefix": "yes", "SyslogPriority": "30", "SystemCallErrorNumber": "0", "TTYReset": "no", "TTYVHangup": "no", "TTYVTDisallocate": "no", "TasksAccounting": "no", "TasksCurrent": "18446744073709551615", "TasksMax": "18446744073709551615", "TimeoutStartUSec": "1min 30s", "TimeoutStopUSec": "1min 30s", "TimerSlackNSec 2019-11-25T09:11:46.082 INFO:teuthology.task.ansible.out:": "50000", "Transient": "no", "Type": "forking", "UMask": "0022", "UnitFilePreset": "disabled", "UnitFileState": "enabled", "WantedBy": "multi-user.target", "Wants": "system.slice", "WatchdogTimestampMonotonic": "0", "WatchdogUSec": "0"}} 2019-11-25T09:11:46.124 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:46 +0000 (0:00:00.142) 0:04:17.505 ******* 2019-11-25T09:11:46.306 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-11-25T09:11:46.338 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:46 +0000 (0:00:00.213) 0:04:17.719 ******* 2019-11-25T09:11:47.006 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on rpm based systems.] ********************* 2019-11-25T09:11:47.008 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "msg": "", "rc": 0, "results": ["python2-pip-8.1.2-10.el7.noarch providing python-pip is already installed"]} 2019-11-25T09:11:47.057 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:47 +0000 (0:00:00.719) 0:04:18.438 ******* 2019-11-25T09:11:47.089 INFO:teuthology.task.ansible.out: TASK [testnode : Install python-pip on apt based systems.] ********************* skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:47.133 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:47 +0000 (0:00:00.075) 0:04:18.513 ******* 2019-11-25T09:11:47.258 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** 2019-11-25T09:11:47.258 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-11-25T09:11:47.296 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:47 +0000 (0:00:00.163) 0:04:18.677 ******* 2019-11-25T09:11:47.446 INFO:teuthology.task.ansible.out: TASK [testnode : Create the .pip directory for the teuthology user.] *********** 2019-11-25T09:11:47.448 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 4096, "state": "directory", "uid": 1000} 2019-11-25T09:11:47.484 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:47 +0000 (0:00:00.187) 0:04:18.865 ******* 2019-11-25T09:11:48.444 INFO:teuthology.task.ansible.out: TASK [testnode : Create pip.conf and configure it to use our mirror] *********** ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "checksum": "959d37ab30545da205b389a6b7561b225a478a06", "dest": "/home/ubuntu/.pip/pip.conf", "gid": 1215, "group": "ubuntu", "mode": "0644", "owner": "ubuntu", "path": "/home/ubuntu/.pip/pip.conf", "secontext": "unconfined_u:object_r:user_home_t:s0", "size": 73, "state": "file", "uid": 1000} 2019-11-25T09:11:48.480 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:48 +0000 (0:00:00.993) 0:04:19.858 ******* 2019-11-25T09:11:49.081 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-11-25T09:11:49.124 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:49 +0000 (0:00:00.642) 0:04:20.501 ******* 2019-11-25T09:11:49.144 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-11-25T09:11:49.145 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:49.187 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:49 +0000 (0:00:00.063) 0:04:20.565 ******* 2019-11-25T09:11:49.212 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-11-25T09:11:49.261 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:49 +0000 (0:00:00.076) 0:04:20.642 ******* 2019-11-25T09:11:49.592 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** ok: [smithi081.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-11-25T09:11:49.626 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:49 +0000 (0:00:00.364) 0:04:21.006 ******* 2019-11-25T09:11:49.652 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:49.691 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:49 +0000 (0:00:00.064) 0:04:21.071 ******* 2019-11-25T09:11:49.727 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** 2019-11-25T09:11:49.729 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-11-25T09:11:49.765 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:49 +0000 (0:00:00.073) 0:04:21.145 ******* 2019-11-25T09:11:49.790 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-11-25T09:11:49.792 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:49.829 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:49 +0000 (0:00:00.063) 0:04:21.209 ******* 2019-11-25T09:11:50.013 INFO:teuthology.task.ansible.out: TASK [testnode : Install packages via pip] ************************************* 2019-11-25T09:11:50.014 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "cmd": ["/bin/pip2", "install", "remoto>=0.0.35"], "name": ["remoto>=0.0.35"], "requirements": null, "state": "present", "stderr": "You are using pip version 8.1.2, however version 19.3.1 is available.\nYou should consider upgrading via the 'pip install --upgrade pip' command.\n", "stderr_lines": ["You are using pip version 8.1.2, however version 19.3.1 is available.", "You should consider upgrading via the 'pip install --upgrade pip' command."], "stdout": "Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages\nRequirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)\nRequirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)\n", "stdout_lines": ["Requirement already satisfied (use --upgrade to upgrade): remoto>=0.0.35 in /usr/lib/python2.7/site-packages", "Requirement already satisfied (use --upgrade to upgrade): execnet in /usr/lib/python2.7/site-packages (from remoto>=0.0.35)", "Requirement already satisfied (use --upgrade to upgrade): apipkg>=1.4 in /usr/lib/python2.7/site-packages (from execnet->remoto>=0.0.35)"], "version": null, "virtualenv": null} 2019-11-25T09:11:50.060 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.232) 0:04:21.441 ******* 2019-11-25T09:11:50.089 INFO:teuthology.task.ansible.out: TASK [testnode : Purge resolvconf] ********************************************* 2019-11-25T09:11:50.090 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.130 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.070) 0:04:21.511 ******* 2019-11-25T09:11:50.148 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-11-25T09:11:50.149 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-11-25T09:11:50.196 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.065) 0:04:21.576 ******* 2019-11-25T09:11:50.202 INFO:teuthology.task.ansible.out: TASK [testnode : Set interface] ************************************************ 2019-11-25T09:11:50.204 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"interface": "enp3s0f1"}, "changed": false} 2019-11-25T09:11:50.241 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.045) 0:04:21.622 ******* 2019-11-25T09:11:50.535 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* changed: [smithi081.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-11-25T09:11:50.574 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.333) 0:04:21.955 ******* 2019-11-25T09:11:50.580 INFO:teuthology.task.ansible.out: TASK [testnode : Check for presence of /etc/network/interfaces] **************** 2019-11-25T09:11:50.581 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => {"changed": false, "stat": {"exists": false}} 2019-11-25T09:11:50.619 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.044) 0:04:22.000 ******* 2019-11-25T09:11:50.626 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-11-25T09:11:50.627 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.665 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.045) 0:04:22.046 ******* 2019-11-25T09:11:50.673 INFO:teuthology.task.ansible.out: TASK [testnode : Rewrite /etc/network/interfaces to use dhcp] ****************** 2019-11-25T09:11:50.675 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.726 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.060) 0:04:22.107 ******* 2019-11-25T09:11:50.732 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* skipping: [smithi081.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.734 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.772 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.046) 0:04:22.153 ******* 2019-11-25T09:11:50.779 INFO:teuthology.task.ansible.out: TASK [testnode : Set bounce_interface if we just enabled dhcp] ***************** ok: [smithi153.front.sepia.ceph.com] => {"ansible_facts": {"bounce_interface": false}, "changed": false} 2019-11-25T09:11:50.833 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.060) 0:04:22.214 ******* 2019-11-25T09:11:50.839 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-11-25T09:11:50.840 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.842 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.843 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi081.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.885 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.052) 0:04:22.266 ******* 2019-11-25T09:11:50.891 INFO:teuthology.task.ansible.out: TASK [testnode : ifdown and ifup] ********************************************** 2019-11-25T09:11:50.893 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.931 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.045) 0:04:22.312 ******* 2019-11-25T09:11:50.937 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:11:50.938 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:50.979 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:50 +0000 (0:00:00.047) 0:04:22.360 ******* 2019-11-25T09:11:51.006 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-11-25T09:11:51.008 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.045 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.065) 0:04:22.425 ******* 2019-11-25T09:11:51.072 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** 2019-11-25T09:11:51.073 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.112 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.066) 0:04:22.492 ******* 2019-11-25T09:11:51.139 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-11-25T09:11:51.140 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.179 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.067) 0:04:22.560 ******* 2019-11-25T09:11:51.206 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-11-25T09:11:51.208 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.268 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.088) 0:04:22.648 ******* 2019-11-25T09:11:51.277 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure lab_domain is in search domains in /etc/resolv.conf] *** 2019-11-25T09:11:51.278 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line replaced"} 2019-11-25T09:11:51.320 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.051) 0:04:22.700 ******* 2019-11-25T09:11:51.325 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-11-25T09:11:51.327 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.372 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.052) 0:04:22.753 ******* 2019-11-25T09:11:51.410 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-11-25T09:11:51.412 INFO:teuthology.task.ansible.out:skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.453 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.080) 0:04:22.833 ******* 2019-11-25T09:11:51.479 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi081.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.517 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.063) 0:04:22.897 ******* 2019-11-25T09:11:51.654 INFO:teuthology.task.ansible.out: TASK [testnode : Ensure domain is set in /etc/resolv.conf] ********************* 2019-11-25T09:11:51.656 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"backup": "", "changed": true, "msg": "line added"} 2019-11-25T09:11:51.692 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.175) 0:04:23.073 ******* 2019-11-25T09:11:51.719 INFO:teuthology.task.ansible.out: TASK [testnode : Make sure ethtool is installed] ******************************* 2019-11-25T09:11:51.721 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.769 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.065) 0:04:23.138 ******* 2019-11-25T09:11:51.795 INFO:teuthology.task.ansible.out: TASK [testnode : grep ethtool for secondary NIC MAC address] ******************* 2019-11-25T09:11:51.797 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=enp3s0f1) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.798 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=eno1) => {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.809 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=eno2) => {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.810 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=lo) => {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.820 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item=enp3s0f0) => {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.859 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.101) 0:04:23.240 ******* 2019-11-25T09:11:51.875 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-11-25T09:11:51.876 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.013189", "end": "2019-11-25 09:11:51.815794", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-11-25 09:11:51.802605", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:51.898 INFO:teuthology.task.ansible.out: TASK [testnode : Define net_to_configure var] ********************************** 2019-11-25T09:11:51.900 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} skipping: [smithi153.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno1', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno1", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.911 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'eno2', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "eno2", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.923 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'lo', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "lo", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.924 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => (item={'skip_reason': u'Conditional result was False', 'item': u'enp3s0f0', 'skipped': True, 'ansible_loop_var': u'item', 'changed': False}) => {"ansible_loop_var": "item", "changed": false, "item": {"ansible_loop_var": "item", "changed": false, "item": "enp3s0f0", "skip_reason": "Conditional result was False", "skipped": true}, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:51.964 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:51 +0000 (0:00:00.104) 0:04:23.344 ******* 2019-11-25T09:11:51.990 INFO:teuthology.task.ansible.out: TASK [testnode : set_fact] ***************************************************** 2019-11-25T09:11:51.995 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:52.031 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:52 +0000 (0:00:00.066) 0:04:23.411 ******* 2019-11-25T09:11:52.058 INFO:teuthology.task.ansible.out: TASK [testnode : Check if {{ nic_to_configure }} is 10Gb] ********************** 2019-11-25T09:11:52.060 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:52.095 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:52 +0000 (0:00:00.064) 0:04:23.476 ******* 2019-11-25T09:11:52.122 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU to 9000 if 10Gb] ************************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:52.163 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:52 +0000 (0:00:00.067) 0:04:23.544 ******* 2019-11-25T09:11:52.190 INFO:teuthology.task.ansible.out: TASK [testnode : Write Ubuntu network config for {{ nic_to_configure }}] ******* 2019-11-25T09:11:52.195 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** ok: [smithi081.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007754", "end": "2019-11-25 09:11:52.143647", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-11-25 09:11:52.135893", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:52.235 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:52 +0000 (0:00:00.072) 0:04:23.616 ******* 2019-11-25T09:11:52.262 INFO:teuthology.task.ansible.out: TASK [testnode : Set MTU on Ubuntu] ******************************************** 2019-11-25T09:11:52.264 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:52.302 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:52 +0000 (0:00:00.066) 0:04:23.682 ******* 2019-11-25T09:11:52.332 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:52.370 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:52 +0000 (0:00:00.068) 0:04:23.751 ******* 2019-11-25T09:11:52.397 INFO:teuthology.task.ansible.out: TASK [testnode : Write RHEL/CentOS network config for {{ nic_to_configure }}] *** 2019-11-25T09:11:52.399 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:52.441 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:52 +0000 (0:00:00.070) 0:04:23.821 ******* 2019-11-25T09:11:52.480 INFO:teuthology.task.ansible.out: TASK [testnode : Bounce {{ nic_to_configure }}] ******************************** 2019-11-25T09:11:52.481 INFO:teuthology.task.ansible.out:skipping: [smithi153.front.sepia.ceph.com] => {"changed": false, "skip_reason": "Conditional result was False"} 2019-11-25T09:11:52.525 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:52 +0000 (0:00:00.084) 0:04:23.906 ******* 2019-11-25T09:11:52.532 INFO:teuthology.task.ansible.out: TASK [testnode : List any leftover Ceph artifacts from previous jobs] ********** 2019-11-25T09:11:52.533 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.007455", "end": "2019-11-25 09:11:52.459804", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-11-25 09:11:52.452349", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:52.879 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=/var/run/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/run/ -name \"*ceph*\"", "delta": "0:00:00.016334", "end": "2019-11-25 09:11:52.832091", "failed_when_result": false, "item": "/var/run/", "rc": 0, "start": "2019-11-25 09:11:52.815757", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:53.182 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=/etc/systemd/system/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/systemd/system/ -name \"*ceph*\"", "delta": "0:00:00.007566", "end": "2019-11-25 09:11:53.146400", "failed_when_result": false, "item": "/etc/systemd/system/", "rc": 0, "start": "2019-11-25 09:11:53.138834", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:53.486 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=/etc/ceph) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /etc/ceph -name \"*ceph*\"", "delta": "0:00:00.006710", "end": "2019-11-25 09:11:53.445823", "failed_when_result": false, "item": "/etc/ceph", "msg": "non-zero return code", "rc": 1, "start": "2019-11-25 09:11:53.439113", "stderr": "find: ‘/etc/ceph’: No such file or directory", "stderr_lines": ["find: ‘/etc/ceph’: No such file or directory"], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:53.835 INFO:teuthology.task.ansible.out:ok: [smithi153.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:00.035235", "end": "2019-11-25 09:11:53.784448", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-11-25 09:11:53.749213", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:53.871 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:53 +0000 (0:00:01.345) 0:04:25.252 ******* 2019-11-25T09:11:53.880 INFO:teuthology.task.ansible.out:ok: [smithi081.front.sepia.ceph.com] => (item=/var/log/) => {"ansible_loop_var": "item", "changed": false, "cmd": "find /var/log/ -name \"*ceph*\"", "delta": "0:00:01.036664", "end": "2019-11-25 09:11:53.813842", "failed_when_result": false, "item": "/var/log/", "rc": 0, "start": "2019-11-25 09:11:52.777178", "stderr": "", "stderr_lines": [], "stdout": "", "stdout_lines": []} 2019-11-25T09:11:53.922 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:53 +0000 (0:00:00.050) 0:04:25.302 ******* 2019-11-25T09:11:53.927 INFO:teuthology.task.ansible.out: TASK [testnode : Delete any leftover Ceph artifacts from previous jobs] ******** 2019-11-25T09:11:53.965 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:53 +0000 (0:00:00.043) 0:04:25.346 ******* 2019-11-25T09:11:54.009 INFO:teuthology.task.ansible.out:Monday 25 November 2019 09:11:54 +0000 (0:00:00.044) 0:04:25.390 ******* 2019-11-25T09:11:54.296 INFO:teuthology.task.ansible.out: TASK [testnode : Touch /ceph-qa-ready] ***************************************** 2019-11-25T09:11:54.297 INFO:teuthology.task.ansible.out:changed: [smithi153.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-11-25T09:11:54.374 INFO:teuthology.task.ansible.out:changed: [smithi081.front.sepia.ceph.com] => {"changed": true, "dest": "/ceph-qa-ready", "gid": 0, "group": "root", "mode": "0644", "owner": "root", "secontext": "unconfined_u:object_r:etc_runtime_t:s0", "size": 0, "state": "file", "uid": 0} 2019-11-25T09:11:54.466 INFO:teuthology.task.ansible.out: PLAY [cobbler] ***************************************************************** skipping: no hosts matched 2019-11-25T09:11:54.468 INFO:teuthology.task.ansible.out: PLAY [paddles] ***************************************************************** 2019-11-25T09:11:54.469 INFO:teuthology.task.ansible.out:skipping: no hosts matched 2019-11-25T09:11:54.471 INFO:teuthology.task.ansible.out: PLAY [pulpito] ***************************************************************** skipping: no hosts matched PLAY RECAP ********************************************************************* smithi081.front.sepia.ceph.com : ok=121 changed=39 unreachable=0 failed=0 skipped=159 rescued=0 ignored=3 smithi153.front.sepia.ceph.com : ok=121 changed=39 unreachable=0 failed=0 skipped=159 rescued=0 ignored=3 Monday 25 November 2019 09:11:54 +0000 (0:00:00.461) 0:04:25.851 ******* =============================================================================== 2019-11-25T09:11:54.472 INFO:teuthology.task.ansible.out:users : Create all admin users with sudo access. ----------------------- 39.17s users : Update authorized_keys using the keys repo --------------------- 37.58s testnode : Install packages -------------------------------------------- 17.15s common : Update yum cache ---------------------------------------------- 11.24s testnode : Ensure ceph packages are not present. ------------------------ 9.96s common : nrpe - Remove SELinux policy package --------------------------- 8.32s common : nrpe - Load SELinux policy package ----------------------------- 8.18s users : Remove revoked users -------------------------------------------- 7.69s testnode : Zap all non-root disks --------------------------------------- 7.55s common : Make sure smartmontools is installed --------------------------- 3.49s Gathering Facts --------------------------------------------------------- 3.29s common : Install nrpe package and dependencies (RHEL/CentOS) ------------ 3.26s ansible-managed : Add authorized keys for the ansible user. ------------- 2.82s testnode : Ensure ceph dependency packages are not present. ------------- 2.69s common : Upload megacli and cli64 for raid monitoring and smart.pl to /usr/sbin/. --- 2.63s common : nrpe - Ensure SELinux policy is up to date --------------------- 2.39s common : Upload custom netsaint scripts for raid/disk/smart/monitoring to /usr/libexec/. --- 2.37s testnode : Rebuild rpmdb ------------------------------------------------ 2.15s testnode : Blow away lingering OSD data and FSIDs ----------------------- 1.63s testnode : Create logical volume(s) ------------------------------------- 1.63s 2019-11-25T09:11:54.734 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@smithi153.front.sepia.ceph.com'), Remote(name='ubuntu@smithi081.front.sepia.ceph.com')] 2019-11-25T09:11:54.736 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-25T09:11:54.737 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi153.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:11:54.837 INFO:teuthology.orchestra.run.smithi153:> true 2019-11-25T09:11:55.221 INFO:teuthology.orchestra.remote:Trying to reconnect to host 2019-11-25T09:11:55.222 DEBUG:teuthology.orchestra.connection:{'username': 'ubuntu', 'hostname': 'smithi081.front.sepia.ceph.com', 'timeout': 60} 2019-11-25T09:11:55.322 INFO:teuthology.orchestra.run.smithi081:> true 2019-11-25T09:11:55.683 INFO:teuthology.run_tasks:Running task clock... 2019-11-25T09:11:55.709 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2019-11-25T09:11:55.710 INFO:teuthology.orchestra.run.smithi153:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-11-25T09:11:55.850 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-11-25T09:12:05.928 INFO:teuthology.orchestra.run.smithi153.stdout:ntpd: time slew +0.001233s 2019-11-25T09:12:05.965 INFO:teuthology.orchestra.run.smithi153.stderr:Failed to start ntp.service: Unit not found. 2019-11-25T09:12:06.097 INFO:teuthology.orchestra.run.smithi153.stdout: remote refid st t when poll reach delay offset jitter 2019-11-25T09:12:06.097 INFO:teuthology.orchestra.run.smithi153.stdout:============================================================================== 2019-11-25T09:12:06.097 INFO:teuthology.orchestra.run.smithi153.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-25T09:12:06.097 INFO:teuthology.orchestra.run.smithi153.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-25T09:12:06.098 INFO:teuthology.orchestra.run.smithi153.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-25T09:12:06.098 INFO:teuthology.orchestra.run.smithi153.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-25T09:12:06.099 INFO:teuthology.orchestra.run.smithi081:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2019-11-25T09:12:06.239 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2019-11-25T09:12:16.327 INFO:teuthology.orchestra.run.smithi081.stdout:ntpd: time slew +0.002912s 2019-11-25T09:12:16.367 INFO:teuthology.orchestra.run.smithi081.stderr:Failed to start ntp.service: Unit not found. 2019-11-25T09:12:16.477 INFO:teuthology.orchestra.run.smithi081.stdout: remote refid st t when poll reach delay offset jitter 2019-11-25T09:12:16.478 INFO:teuthology.orchestra.run.smithi081.stdout:============================================================================== 2019-11-25T09:12:16.478 INFO:teuthology.orchestra.run.smithi081.stdout: hv01.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-25T09:12:16.478 INFO:teuthology.orchestra.run.smithi081.stdout: hv02.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-25T09:12:16.478 INFO:teuthology.orchestra.run.smithi081.stdout: hv03.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-25T09:12:16.479 INFO:teuthology.orchestra.run.smithi081.stdout: hv04.front.sepi .INIT. 16 u - 64 0 0.000 0.000 0.000 2019-11-25T09:12:16.479 INFO:teuthology.run_tasks:Running task install... 2019-11-25T09:12:16.494 DEBUG:teuthology.task.install:project ceph 2019-11-25T09:12:16.494 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'notcmalloc', 'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343'}} 2019-11-25T09:12:16.494 DEBUG:teuthology.task.install:config {'flavor': 'notcmalloc', 'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343'} 2019-11-25T09:12:16.494 INFO:teuthology.task.install:Using flavor: notcmalloc 2019-11-25T09:12:16.503 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'ceph-daemon', 'ceph-mds', 'ceph-mgr', 'ceph-common', 'ceph-fuse', 'ceph-test', 'radosgw', 'python-ceph', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-daemon', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-cloud', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-ssh', 'ceph-fuse', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python-ceph', 'rbd-fuse']} 2019-11-25T09:12:16.503 INFO:teuthology.task.install:extra packages: [] 2019-11-25T09:12:16.503 INFO:teuthology.orchestra.run.smithi153:> sudo lsb_release -is 2019-11-25T09:12:16.672 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS 2019-11-25T09:12:16.674 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-25T09:12:16.674 INFO:teuthology.orchestra.run.smithi081:> sudo lsb_release -is 2019-11-25T09:12:16.675 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-11-25T09:12:16.675 INFO:teuthology.orchestra.run.smithi153:> sudo lsb_release -is 2019-11-25T09:12:16.818 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS 2019-11-25T09:12:16.820 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-25T09:12:16.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=12b9659be24ae0e3d791f80d81504ab2770c1343 2019-11-25T09:12:16.840 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS 2019-11-25T09:12:16.841 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-25T09:12:16.841 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343', 'exclude_packages': [], 'debuginfo': None, 'install_ceph_packages': True, 'wait_for_package': False, 'downgrade_packages': [], 'project': 'ceph', 'tag': None, 'extra_packages': [], 'branch': None, 'packages': {}, 'repos_only': False, 'extra_system_packages': [], 'flavor': 'notcmalloc', 'extras': None} 2019-11-25T09:12:16.841 INFO:teuthology.orchestra.run.smithi081:> sudo lsb_release -is 2019-11-25T09:12:16.956 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS 2019-11-25T09:12:16.959 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-25T09:12:16.959 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=12b9659be24ae0e3d791f80d81504ab2770c1343 2019-11-25T09:12:16.997 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuval-add-kafka-security2/12b9659be24ae0e3d791f80d81504ab2770c1343/centos/7/flavors/notcmalloc/ 2019-11-25T09:12:16.998 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7614.g12b9659 2019-11-25T09:12:17.143 INFO:teuthology.orchestra.run.smithi153:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-11-25T09:12:17.147 INFO:teuthology.task.install.rpm:Pulling from https://4.chacra.ceph.com/r/ceph/wip-yuval-add-kafka-security2/12b9659be24ae0e3d791f80d81504ab2770c1343/centos/7/flavors/notcmalloc/ 2019-11-25T09:12:17.147 INFO:teuthology.task.install.rpm:Package version is 15.0.0-7614.g12b9659 2019-11-25T09:12:17.274 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-11-25T09:12:17.274 INFO:teuthology.orchestra.run.smithi153:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/12b9659be24ae0e3d791f80d81504ab2770c1343/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-11-25T09:12:17.287 INFO:teuthology.orchestra.run.smithi081:> sudo sh -c 'cat > /etc/yum.repos.d/ceph.repo' 2019-11-25T09:12:17.365 INFO:teuthology.orchestra.run.smithi153:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-11-25T09:12:17.406 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-daemon, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-cloud, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-ssh, ceph-fuse, libcephfs2, libcephfs-devel, librados2, librbd1, python-ceph, rbd-fuse on remote rpm x86_64 2019-11-25T09:12:17.407 INFO:teuthology.orchestra.run.smithi081:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/12b9659be24ae0e3d791f80d81504ab2770c1343/;g' /etc/yum.repos.d/ceph.repo ; fi 2019-11-25T09:12:17.445 INFO:teuthology.orchestra.run.smithi153:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-11-25T09:12:17.498 INFO:teuthology.orchestra.run.smithi081:> test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2019-11-25T09:12:17.513 INFO:teuthology.orchestra.run.smithi153.stdout:check_obsoletes = 1 2019-11-25T09:12:17.515 INFO:teuthology.orchestra.run.smithi153:> sudo yum clean all 2019-11-25T09:12:17.577 INFO:teuthology.orchestra.run.smithi081:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2019-11-25T09:12:17.653 INFO:teuthology.orchestra.run.smithi081.stdout:check_obsoletes = 1 2019-11-25T09:12:17.655 INFO:teuthology.orchestra.run.smithi081:> sudo yum clean all 2019-11-25T09:12:17.666 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:12:17.704 INFO:teuthology.orchestra.run.smithi153.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-11-25T09:12:17.704 INFO:teuthology.orchestra.run.smithi153.stdout: : lab-extras updates 2019-11-25T09:12:17.749 INFO:teuthology.orchestra.run.smithi153.stdout:Cleaning up list of fastest mirrors 2019-11-25T09:12:17.749 INFO:teuthology.orchestra.run.smithi153.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-11-25T09:12:17.762 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-radosgw 2019-11-25T09:12:17.820 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:12:17.859 INFO:teuthology.orchestra.run.smithi081.stdout:Cleaning repos: base centos7-fcgi-ceph ceph ceph-noarch ceph-source epel extras 2019-11-25T09:12:17.859 INFO:teuthology.orchestra.run.smithi081.stdout: : lab-extras updates 2019-11-25T09:12:17.903 INFO:teuthology.orchestra.run.smithi081.stdout:Cleaning up list of fastest mirrors 2019-11-25T09:12:17.903 INFO:teuthology.orchestra.run.smithi081.stdout:Other repos take up 1.5 M of disk space (use --verbose for details) 2019-11-25T09:12:17.916 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-radosgw 2019-11-25T09:12:17.924 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:12:17.966 INFO:teuthology.orchestra.run.smithi153.stdout:Determining fastest mirrors 2019-11-25T09:12:18.157 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:12:18.200 INFO:teuthology.orchestra.run.smithi081.stdout:Determining fastest mirrors 2019-11-25T09:12:18.533 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:12:18.534 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:12:18.534 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:12:18.535 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:12:18.893 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:12:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:12:18.894 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:12:18.895 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:12:26.322 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:12:26.674 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:12:26.674 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:12:26.674 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.687 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.799 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.801 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.802 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.803 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.803 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.805 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.806 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.813 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.825 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:12:26.826 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-base.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.827 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.827 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.828 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.828 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.828 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.828 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.828 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-11-25T09:12:26.828 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-11-25T09:12:26.828 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-11-25T09:12:26.829 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package librados2.x86_64 2:15.0.0-7614.g12b9659.el7 will be an update 2019-11-25T09:12:26.829 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-11-25T09:12:26.829 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package librgw2.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.840 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:12:26.840 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-common.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.841 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.842 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.842 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.842 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.843 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:26.843 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-11-25T09:12:26.844 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.844 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-11-25T09:12:26.844 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package librbd1.x86_64 2:15.0.0-7614.g12b9659.el7 will be an update 2019-11-25T09:12:26.845 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-11-25T09:12:26.846 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-11-25T09:12:26.847 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-11-25T09:12:26.847 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:12:26.847 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-11-25T09:12:26.848 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.850 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.850 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.850 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.851 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-11-25T09:12:26.851 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-rados.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.852 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-rbd.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.852 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-rgw.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:26.852 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-11-25T09:12:27.197 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:12:27.543 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:12:27.544 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:12:27.544 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-radosgw.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.556 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: librados2 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.664 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: ceph-base = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.666 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: ceph-selinux = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.667 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: librgw2 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.668 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: librados.so.2(LIBRADOS_14.2.0)(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.669 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: librdkafka.so.1()(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.671 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libceph-common.so.2()(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.671 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: librabbitmq.so.4()(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.673 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: liboath.so.0()(64bit) for package: 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.674 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:12:27.674 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-base.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.682 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: librbd1 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.683 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: ceph-common = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.685 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libcephfs2 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.686 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libleveldb.so.1()(64bit) for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.687 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: liblttng-ust.so.0()(64bit) for package: 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.688 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-selinux.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.690 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package liboath.x86_64 0:2.6.2-1.el7 will be installed 2019-11-25T09:12:27.691 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package librabbitmq.x86_64 0:0.8.0-2.el7 will be installed 2019-11-25T09:12:27.691 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package librados2.x86_64 1:10.2.5-4.el7 will be updated 2019-11-25T09:12:27.692 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package librados2.x86_64 2:15.0.0-7614.g12b9659.el7 will be an update 2019-11-25T09:12:27.693 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package librdkafka.x86_64 0:0.11.5-1.el7 will be installed 2019-11-25T09:12:27.694 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package librgw2.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.695 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:12:27.695 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-common.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.699 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-cephfs = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.701 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-rbd = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.702 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-ceph-argparse = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.702 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-rados = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.703 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-rgw = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.704 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-ceph-common = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.705 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libradosstriper1 = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.706 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-prettytable for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.707 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libradosstriper.so.1()(64bit) for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.707 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libbabeltrace.so.1()(64bit) for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.709 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libbabeltrace-ctf.so.1()(64bit) for package: 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:27.709 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package leveldb.x86_64 0:1.12.0-11.el7 will be installed 2019-11-25T09:12:27.709 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package libcephfs2.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.709 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package librbd1.x86_64 1:10.2.5-4.el7 will be updated 2019-11-25T09:12:27.710 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package librbd1.x86_64 2:15.0.0-7614.g12b9659.el7 will be an update 2019-11-25T09:12:27.710 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package lttng-ust.x86_64 0:2.4.1-4.el7 will be installed 2019-11-25T09:12:27.711 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: liburcu-bp.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-11-25T09:12:27.712 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: liburcu-cds.so.1()(64bit) for package: lttng-ust-2.4.1-4.el7.x86_64 2019-11-25T09:12:27.712 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:12:27.712 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package libbabeltrace.x86_64 0:1.2.4-3.el7 will be installed 2019-11-25T09:12:27.713 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package libradosstriper1.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.714 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-ceph-argparse.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.715 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-ceph-common.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.715 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-cephfs.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.716 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-prettytable.noarch 0:0.7.2-3.el7 will be installed 2019-11-25T09:12:27.716 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-rados.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.717 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-rbd.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.717 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-rgw.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:27.717 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package userspace-rcu.x86_64 0:0.7.16-1.el7 will be installed 2019-11-25T09:12:27.728 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:12:27.832 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:27.833 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:12:27.833 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:27.834 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:12:27.834 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository 2019-11-25T09:12:27.835 INFO:teuthology.orchestra.run.smithi153.stdout: Size 2019-11-25T09:12:27.835 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:12:27.835 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:12:27.835 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw x86_64 2:15.0.0-7614.g12b9659.el7 ceph 6.2 M 2019-11-25T09:12:27.835 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:12:27.835 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base x86_64 2:15.0.0-7614.g12b9659.el7 ceph 5.6 M 2019-11-25T09:12:27.836 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common x86_64 2:15.0.0-7614.g12b9659.el7 ceph 20 M 2019-11-25T09:12:27.836 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux x86_64 2:15.0.0-7614.g12b9659.el7 ceph 20 k 2019-11-25T09:12:27.836 INFO:teuthology.orchestra.run.smithi153.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-11-25T09:12:27.836 INFO:teuthology.orchestra.run.smithi153.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-11-25T09:12:27.836 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 489 k 2019-11-25T09:12:27.836 INFO:teuthology.orchestra.run.smithi153.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-11-25T09:12:27.837 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-11-25T09:12:27.837 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 374 k 2019-11-25T09:12:27.837 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-11-25T09:12:27.837 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 5.4 M 2019-11-25T09:12:27.837 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-11-25T09:12:27.837 INFO:teuthology.orchestra.run.smithi153.stdout: python-ceph-argparse x86_64 2:15.0.0-7614.g12b9659.el7 ceph 35 k 2019-11-25T09:12:27.838 INFO:teuthology.orchestra.run.smithi153.stdout: python-ceph-common x86_64 2:15.0.0-7614.g12b9659.el7 ceph 42 k 2019-11-25T09:12:27.838 INFO:teuthology.orchestra.run.smithi153.stdout: python-cephfs x86_64 2:15.0.0-7614.g12b9659.el7 ceph 102 k 2019-11-25T09:12:27.838 INFO:teuthology.orchestra.run.smithi153.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-11-25T09:12:27.838 INFO:teuthology.orchestra.run.smithi153.stdout: python-rados x86_64 2:15.0.0-7614.g12b9659.el7 ceph 209 k 2019-11-25T09:12:27.838 INFO:teuthology.orchestra.run.smithi153.stdout: python-rbd x86_64 2:15.0.0-7614.g12b9659.el7 ceph 199 k 2019-11-25T09:12:27.838 INFO:teuthology.orchestra.run.smithi153.stdout: python-rgw x86_64 2:15.0.0-7614.g12b9659.el7 ceph 77 k 2019-11-25T09:12:27.838 INFO:teuthology.orchestra.run.smithi153.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-11-25T09:12:27.839 INFO:teuthology.orchestra.run.smithi153.stdout:Updating for dependencies: 2019-11-25T09:12:27.839 INFO:teuthology.orchestra.run.smithi153.stdout: librados2 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 3.5 M 2019-11-25T09:12:27.839 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 1.8 M 2019-11-25T09:12:27.839 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:27.839 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:12:27.839 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:12:27.840 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+20 Dependent packages) 2019-11-25T09:12:27.840 INFO:teuthology.orchestra.run.smithi153.stdout:Upgrade ( 2 Dependent packages) 2019-11-25T09:12:27.840 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:27.840 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 45 M 2019-11-25T09:12:27.850 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:12:27.850 INFO:teuthology.orchestra.run.smithi153.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-25T09:12:28.568 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:12:28.653 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:28.653 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:12:28.687 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:28.687 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository 2019-11-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: Size 2019-11-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw x86_64 2:15.0.0-7614.g12b9659.el7 ceph 6.2 M 2019-11-25T09:12:28.688 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:12:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base x86_64 2:15.0.0-7614.g12b9659.el7 ceph 5.6 M 2019-11-25T09:12:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common x86_64 2:15.0.0-7614.g12b9659.el7 ceph 20 M 2019-11-25T09:12:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux x86_64 2:15.0.0-7614.g12b9659.el7 ceph 20 k 2019-11-25T09:12:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: leveldb x86_64 1.12.0-11.el7 epel 161 k 2019-11-25T09:12:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: libbabeltrace x86_64 1.2.4-3.el7 epel 147 k 2019-11-25T09:12:28.689 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 489 k 2019-11-25T09:12:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: liboath x86_64 2.6.2-1.el7 epel 51 k 2019-11-25T09:12:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq x86_64 0.8.0-2.el7 base 37 k 2019-11-25T09:12:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 374 k 2019-11-25T09:12:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka x86_64 0.11.5-1.el7 epel 368 k 2019-11-25T09:12:28.690 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 5.4 M 2019-11-25T09:12:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust x86_64 2.4.1-4.el7 epel 176 k 2019-11-25T09:12:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: python-ceph-argparse x86_64 2:15.0.0-7614.g12b9659.el7 ceph 35 k 2019-11-25T09:12:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: python-ceph-common x86_64 2:15.0.0-7614.g12b9659.el7 ceph 42 k 2019-11-25T09:12:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: python-cephfs x86_64 2:15.0.0-7614.g12b9659.el7 ceph 102 k 2019-11-25T09:12:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: python-prettytable noarch 0.7.2-3.el7 base 37 k 2019-11-25T09:12:28.691 INFO:teuthology.orchestra.run.smithi081.stdout: python-rados x86_64 2:15.0.0-7614.g12b9659.el7 ceph 209 k 2019-11-25T09:12:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: python-rbd x86_64 2:15.0.0-7614.g12b9659.el7 ceph 199 k 2019-11-25T09:12:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: python-rgw x86_64 2:15.0.0-7614.g12b9659.el7 ceph 77 k 2019-11-25T09:12:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: userspace-rcu x86_64 0.7.16-1.el7 epel 73 k 2019-11-25T09:12:28.692 INFO:teuthology.orchestra.run.smithi081.stdout:Updating for dependencies: 2019-11-25T09:12:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: librados2 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 3.5 M 2019-11-25T09:12:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1 x86_64 2:15.0.0-7614.g12b9659.el7 ceph 1.8 M 2019-11-25T09:12:28.692 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:28.693 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:12:28.693 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:12:28.693 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+20 Dependent packages) 2019-11-25T09:12:28.693 INFO:teuthology.orchestra.run.smithi081.stdout:Upgrade ( 2 Dependent packages) 2019-11-25T09:12:28.693 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:28.693 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 45 M 2019-11-25T09:12:28.694 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:12:28.694 INFO:teuthology.orchestra.run.smithi081.stdout:Delta RPMs disabled because /usr/bin/applydeltarpm not installed. 2019-11-25T09:12:29.924 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:12:29.924 INFO:teuthology.orchestra.run.smithi153.stdout:Total 21 MB/s | 45 MB 00:02 2019-11-25T09:12:29.941 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:12:30.016 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:12:30.293 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:12:30.293 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:12:30.931 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-11-25T09:12:31.324 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-11-25T09:12:31.670 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-11-25T09:12:31.848 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:12:31.848 INFO:teuthology.orchestra.run.smithi081.stdout:Total 14 MB/s | 45 MB 00:03 2019-11-25T09:12:31.872 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:12:31.932 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:12:31.984 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:python-ceph-argparse-15.0.0-7614.g12b9659.el7.x86_64 4/25 2019-11-25T09:12:32.225 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:12:32.226 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:12:32.363 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 5/25 2019-11-25T09:12:32.734 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-11-25T09:12:32.870 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : liboath-2.6.2-1.el7.x86_64 1/25 2019-11-25T09:12:33.232 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librabbitmq-0.8.0-2.el7.x86_64 2/25 2019-11-25T09:12:33.525 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : librdkafka-0.11.5-1.el7.x86_64 3/25 2019-11-25T09:12:33.544 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-11-25T09:12:33.831 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:python-ceph-argparse-15.0.0-7614.g12b9659.el7.x86_64 4/25 2019-11-25T09:12:33.940 INFO:teuthology.orchestra.run.smithi153.stdout: Updating : 2:librados2-15.0.0-7614.g12b9659.el7.x86_64 8/25 2019-11-25T09:12:34.151 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : leveldb-1.12.0-11.el7.x86_64 5/25 2019-11-25T09:12:34.318 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:python-rados-15.0.0-7614.g12b9659.el7.x86_64 9/25 2019-11-25T09:12:34.538 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : userspace-rcu-0.7.16-1.el7.x86_64 6/25 2019-11-25T09:12:35.235 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:libcephfs2-15.0.0-7614.g12b9659.el7.x86_64 10/25 2019-11-25T09:12:35.294 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : lttng-ust-2.4.1-4.el7.x86_64 7/25 2019-11-25T09:12:35.629 INFO:teuthology.orchestra.run.smithi081.stdout: Updating : 2:librados2-15.0.0-7614.g12b9659.el7.x86_64 8/25 2019-11-25T09:12:35.787 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:librgw2-15.0.0-7614.g12b9659.el7.x86_64 11/25 2019-11-25T09:12:35.986 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:python-rados-15.0.0-7614.g12b9659.el7.x86_64 9/25 2019-11-25T09:12:36.139 INFO:teuthology.orchestra.run.smithi153.stdout: Updating : 2:librbd1-15.0.0-7614.g12b9659.el7.x86_64 12/25 2019-11-25T09:12:36.374 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:python-rbd-15.0.0-7614.g12b9659.el7.x86_64 13/25 2019-11-25T09:12:36.615 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:python-rgw-15.0.0-7614.g12b9659.el7.x86_64 14/25 2019-11-25T09:12:36.904 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:libcephfs2-15.0.0-7614.g12b9659.el7.x86_64 10/25 2019-11-25T09:12:36.939 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:python-cephfs-15.0.0-7614.g12b9659.el7.x86_64 15/25 2019-11-25T09:12:37.246 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:libradosstriper1-15.0.0-7614.g12b9659.el7.x86_64 16/25 2019-11-25T09:12:37.430 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:librgw2-15.0.0-7614.g12b9659.el7.x86_64 11/25 2019-11-25T09:12:37.566 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-11-25T09:12:37.802 INFO:teuthology.orchestra.run.smithi081.stdout: Updating : 2:librbd1-15.0.0-7614.g12b9659.el7.x86_64 12/25 2019-11-25T09:12:37.882 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:python-ceph-common-15.0.0-7614.g12b9659.el7.x86_64 18/25 2019-11-25T09:12:38.070 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:python-rbd-15.0.0-7614.g12b9659.el7.x86_64 13/25 2019-11-25T09:12:38.383 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:python-rgw-15.0.0-7614.g12b9659.el7.x86_64 14/25 2019-11-25T09:12:38.694 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:python-cephfs-15.0.0-7614.g12b9659.el7.x86_64 15/25 2019-11-25T09:12:38.995 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:libradosstriper1-15.0.0-7614.g12b9659.el7.x86_64 16/25 2019-11-25T09:12:39.272 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-prettytable-0.7.2-3.el7.noarch 17/25 2019-11-25T09:12:39.578 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:python-ceph-common-15.0.0-7614.g12b9659.el7.x86_64 18/25 2019-11-25T09:12:40.385 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 19/25 2019-11-25T09:12:41.512 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 20/25 2019-11-25T09:12:42.066 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : libbabeltrace-1.2.4-3.el7.x86_64 19/25 2019-11-25T09:12:42.111 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 21/25 2019-11-25T09:12:43.245 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 20/25 2019-11-25T09:12:43.784 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 21/25 2019-11-25T09:12:51.550 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-selinux-15.0.0-7614.g12b9659.el7.x86_64 22/25 2019-11-25T09:12:51.926 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 23/25 2019-11-25T09:12:52.172 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-11-25T09:12:52.447 INFO:teuthology.orchestra.run.smithi153.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-11-25T09:12:52.606 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-selinux-15.0.0-7614.g12b9659.el7.x86_64 1/25 2019-11-25T09:12:52.748 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-11-25T09:12:52.907 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:librados2-15.0.0-7614.g12b9659.el7.x86_64 3/25 2019-11-25T09:12:53.033 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-11-25T09:12:53.183 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:python-rgw-15.0.0-7614.g12b9659.el7.x86_64 5/25 2019-11-25T09:12:53.259 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-selinux-15.0.0-7614.g12b9659.el7.x86_64 22/25 2019-11-25T09:12:53.309 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:python-cephfs-15.0.0-7614.g12b9659.el7.x86_64 6/25 2019-11-25T09:12:53.409 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 7/25 2019-11-25T09:12:53.518 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 8/25 2019-11-25T09:12:53.635 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 23/25 2019-11-25T09:12:53.660 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:python-rados-15.0.0-7614.g12b9659.el7.x86_64 9/25 2019-11-25T09:12:53.752 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 10/25 2019-11-25T09:12:53.837 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:python-rbd-15.0.0-7614.g12b9659.el7.x86_64 11/25 2019-11-25T09:12:53.953 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:libcephfs2-15.0.0-7614.g12b9659.el7.x86_64 12/25 2019-11-25T09:12:54.029 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:python-ceph-common-15.0.0-7614.g12b9659.el7.x86_64 13/25 2019-11-25T09:12:54.129 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 14/25 2019-11-25T09:12:54.263 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 15/25 2019-11-25T09:12:54.380 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 16/25 2019-11-25T09:12:54.498 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:librgw2-15.0.0-7614.g12b9659.el7.x86_64 17/25 2019-11-25T09:12:54.624 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7614.g12b9659.el7.x86_64 18/25 2019-11-25T09:12:54.649 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-11-25T09:12:54.749 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:libradosstriper1-15.0.0-7614.g12b9659.el7.x86_64 19/25 2019-11-25T09:12:54.891 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 20/25 2019-11-25T09:12:54.959 INFO:teuthology.orchestra.run.smithi081.stdout: Cleanup : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-11-25T09:12:55.108 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-selinux-15.0.0-7614.g12b9659.el7.x86_64 1/25 2019-11-25T09:12:55.242 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : liboath-2.6.2-1.el7.x86_64 2/25 2019-11-25T09:12:55.351 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:librados2-15.0.0-7614.g12b9659.el7.x86_64 3/25 2019-11-25T09:12:55.460 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : leveldb-1.12.0-11.el7.x86_64 4/25 2019-11-25T09:12:55.568 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:python-rgw-15.0.0-7614.g12b9659.el7.x86_64 5/25 2019-11-25T09:12:55.625 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-11-25T09:12:55.677 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:python-cephfs-15.0.0-7614.g12b9659.el7.x86_64 6/25 2019-11-25T09:12:55.750 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:librbd1-15.0.0-7614.g12b9659.el7.x86_64 22/25 2019-11-25T09:12:55.763 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : libbabeltrace-1.2.4-3.el7.x86_64 7/25 2019-11-25T09:12:55.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 23/25 2019-11-25T09:12:55.869 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-11-25T09:12:55.870 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-base-15.0.0-7614.g12b9659.el7.x86_64 8/25 2019-11-25T09:12:55.996 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:python-rados-15.0.0-7614.g12b9659.el7.x86_64 9/25 2019-11-25T09:12:56.113 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-common-15.0.0-7614.g12b9659.el7.x86_64 10/25 2019-11-25T09:12:56.208 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-11-25T09:12:56.208 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:56.208 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:12:56.209 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-radosgw.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.209 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:56.210 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:12:56.210 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-base.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.210 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-common.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.210 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-selinux.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.210 INFO:teuthology.orchestra.run.smithi153.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-11-25T09:12:56.210 INFO:teuthology.orchestra.run.smithi153.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-11-25T09:12:56.210 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs2.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.211 INFO:teuthology.orchestra.run.smithi153.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-11-25T09:12:56.211 INFO:teuthology.orchestra.run.smithi153.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-11-25T09:12:56.211 INFO:teuthology.orchestra.run.smithi153.stdout: libradosstriper1.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.211 INFO:teuthology.orchestra.run.smithi153.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-11-25T09:12:56.212 INFO:teuthology.orchestra.run.smithi153.stdout: librgw2.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.212 INFO:teuthology.orchestra.run.smithi153.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-11-25T09:12:56.213 INFO:teuthology.orchestra.run.smithi153.stdout: python-ceph-argparse.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.213 INFO:teuthology.orchestra.run.smithi153.stdout: python-ceph-common.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.213 INFO:teuthology.orchestra.run.smithi153.stdout: python-cephfs.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.213 INFO:teuthology.orchestra.run.smithi153.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-11-25T09:12:56.213 INFO:teuthology.orchestra.run.smithi153.stdout: python-rados.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.213 INFO:teuthology.orchestra.run.smithi153.stdout: python-rbd.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.213 INFO:teuthology.orchestra.run.smithi153.stdout: python-rgw.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.213 INFO:teuthology.orchestra.run.smithi153.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-11-25T09:12:56.214 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:56.214 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Updated: 2019-11-25T09:12:56.214 INFO:teuthology.orchestra.run.smithi153.stdout: librados2.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.214 INFO:teuthology.orchestra.run.smithi153.stdout: librbd1.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:56.214 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:56.214 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:12:56.215 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:python-rbd-15.0.0-7614.g12b9659.el7.x86_64 11/25 2019-11-25T09:12:56.330 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-test 2019-11-25T09:12:56.333 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:libcephfs2-15.0.0-7614.g12b9659.el7.x86_64 12/25 2019-11-25T09:12:56.440 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:python-ceph-common-15.0.0-7614.g12b9659.el7.x86_64 13/25 2019-11-25T09:12:56.565 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librdkafka-0.11.5-1.el7.x86_64 14/25 2019-11-25T09:12:56.644 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:12:56.689 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:12:56.690 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:12:56.691 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : lttng-ust-2.4.1-4.el7.x86_64 15/25 2019-11-25T09:12:56.691 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:12:56.692 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:12:56.692 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:12:56.817 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-radosgw-15.0.0-7614.g12b9659.el7.x86_64 16/25 2019-11-25T09:12:56.932 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:librgw2-15.0.0-7614.g12b9659.el7.x86_64 17/25 2019-11-25T09:12:57.018 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:python-ceph-argparse-15.0.0-7614.g12b9659.el7.x86_64 18/25 2019-11-25T09:12:57.110 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:libradosstriper1-15.0.0-7614.g12b9659.el7.x86_64 19/25 2019-11-25T09:12:57.202 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-prettytable-0.7.2-3.el7.noarch 20/25 2019-11-25T09:12:57.286 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : userspace-rcu-0.7.16-1.el7.x86_64 21/25 2019-11-25T09:12:57.340 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:12:57.370 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:librbd1-15.0.0-7614.g12b9659.el7.x86_64 22/25 2019-11-25T09:12:57.496 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : librabbitmq-0.8.0-2.el7.x86_64 23/25 2019-11-25T09:12:57.497 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 1:librbd1-10.2.5-4.el7.x86_64 24/25 2019-11-25T09:12:57.682 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:12:57.683 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:12:57.683 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-test.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:57.700 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:57.730 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:57.731 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:57.732 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:12:57.732 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-11-25T09:12:57.733 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-11-25T09:12:57.733 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-11-25T09:12:57.735 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-11-25T09:12:57.738 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:12:57.738 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-11-25T09:12:57.861 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 1:librados2-10.2.5-4.el7.x86_64 25/25 2019-11-25T09:12:57.862 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:57.862 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:12:57.862 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-radosgw.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.862 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:57.862 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:12:57.862 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-base.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.862 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-common.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.863 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-selinux.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.863 INFO:teuthology.orchestra.run.smithi081.stdout: leveldb.x86_64 0:1.12.0-11.el7 2019-11-25T09:12:57.863 INFO:teuthology.orchestra.run.smithi081.stdout: libbabeltrace.x86_64 0:1.2.4-3.el7 2019-11-25T09:12:57.863 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs2.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.863 INFO:teuthology.orchestra.run.smithi081.stdout: liboath.x86_64 0:2.6.2-1.el7 2019-11-25T09:12:57.863 INFO:teuthology.orchestra.run.smithi081.stdout: librabbitmq.x86_64 0:0.8.0-2.el7 2019-11-25T09:12:57.864 INFO:teuthology.orchestra.run.smithi081.stdout: libradosstriper1.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.864 INFO:teuthology.orchestra.run.smithi081.stdout: librdkafka.x86_64 0:0.11.5-1.el7 2019-11-25T09:12:57.864 INFO:teuthology.orchestra.run.smithi081.stdout: librgw2.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.864 INFO:teuthology.orchestra.run.smithi081.stdout: lttng-ust.x86_64 0:2.4.1-4.el7 2019-11-25T09:12:57.864 INFO:teuthology.orchestra.run.smithi081.stdout: python-ceph-argparse.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.864 INFO:teuthology.orchestra.run.smithi081.stdout: python-ceph-common.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.864 INFO:teuthology.orchestra.run.smithi081.stdout: python-cephfs.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.865 INFO:teuthology.orchestra.run.smithi081.stdout: python-prettytable.noarch 0:0.7.2-3.el7 2019-11-25T09:12:57.865 INFO:teuthology.orchestra.run.smithi081.stdout: python-rados.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.865 INFO:teuthology.orchestra.run.smithi081.stdout: python-rbd.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.865 INFO:teuthology.orchestra.run.smithi081.stdout: python-rgw.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.865 INFO:teuthology.orchestra.run.smithi081.stdout: userspace-rcu.x86_64 0:0.7.16-1.el7 2019-11-25T09:12:57.865 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:57.865 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Updated: 2019-11-25T09:12:57.865 INFO:teuthology.orchestra.run.smithi081.stdout: librados2.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.866 INFO:teuthology.orchestra.run.smithi081.stdout: librbd1.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:12:57.866 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:57.866 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:12:57.990 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-test 2019-11-25T09:12:58.100 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:12:58.290 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:58.325 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository 2019-11-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi153.stdout: Size 2019-11-25T09:12:58.326 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test x86_64 2:15.0.0-7614.g12b9659.el7 ceph 38 M 2019-11-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi153.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-11-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi153.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-11-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi153.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-11-25T09:12:58.327 INFO:teuthology.orchestra.run.smithi153.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-11-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+4 Dependent packages) 2019-11-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 38 M 2019-11-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 134 M 2019-11-25T09:12:58.328 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:12:58.342 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:12:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:12:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:12:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:12:58.343 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:12:58.979 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:12:59.311 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:12:59.312 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:12:59.312 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-test.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:12:59.329 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: xmlstarlet for package: 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:59.358 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: jq for package: 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:59.359 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: socat for package: 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:12:59.360 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:12:59.361 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package jq.x86_64 0:1.5-1.el7 will be installed 2019-11-25T09:12:59.361 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libonig.so.2()(64bit) for package: jq-1.5-1.el7.x86_64 2019-11-25T09:12:59.362 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package socat.x86_64 0:1.7.3.2-2.el7 will be installed 2019-11-25T09:12:59.364 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package xmlstarlet.x86_64 0:1.6.1-1.el7 will be installed 2019-11-25T09:12:59.367 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:12:59.367 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package oniguruma.x86_64 0:5.9.5-3.el7 will be installed 2019-11-25T09:12:59.725 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:12:59.945 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:59.945 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:12:59.946 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:59.946 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:12:59.946 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository 2019-11-25T09:12:59.947 INFO:teuthology.orchestra.run.smithi081.stdout: Size 2019-11-25T09:12:59.947 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:12:59.947 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:12:59.947 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test x86_64 2:15.0.0-7614.g12b9659.el7 ceph 38 M 2019-11-25T09:12:59.947 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:12:59.947 INFO:teuthology.orchestra.run.smithi081.stdout: jq x86_64 1.5-1.el7 epel 153 k 2019-11-25T09:12:59.947 INFO:teuthology.orchestra.run.smithi081.stdout: oniguruma x86_64 5.9.5-3.el7 epel 129 k 2019-11-25T09:12:59.948 INFO:teuthology.orchestra.run.smithi081.stdout: socat x86_64 1.7.3.2-2.el7 base 290 k 2019-11-25T09:12:59.948 INFO:teuthology.orchestra.run.smithi081.stdout: xmlstarlet x86_64 1.6.1-1.el7 epel 62 k 2019-11-25T09:12:59.948 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:59.948 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:12:59.948 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:12:59.948 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+4 Dependent packages) 2019-11-25T09:12:59.948 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:12:59.949 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 38 M 2019-11-25T09:12:59.949 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 134 M 2019-11-25T09:12:59.949 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:13:00.120 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:00.124 INFO:teuthology.orchestra.run.smithi153.stdout:Total 21 MB/s | 38 MB 00:01 2019-11-25T09:13:00.125 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:13:00.140 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:13:00.219 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:13:00.220 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:13:00.871 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-11-25T09:13:01.173 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-11-25T09:13:01.493 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-11-25T09:13:01.780 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:01.781 INFO:teuthology.orchestra.run.smithi081.stdout:Total 21 MB/s | 38 MB 00:01 2019-11-25T09:13:01.784 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:13:01.811 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:13:01.878 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:13:01.878 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:13:02.622 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : xmlstarlet-1.6.1-1.el7.x86_64 1/5 2019-11-25T09:13:02.917 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : socat-1.7.3.2-2.el7.x86_64 2/5 2019-11-25T09:13:03.214 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : oniguruma-5.9.5-3.el7.x86_64 3/5 2019-11-25T09:13:05.532 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-11-25T09:13:05.877 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 5/5 2019-11-25T09:13:05.967 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2019-11-25T09:13:06.059 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 2/5 2019-11-25T09:13:06.143 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : jq-1.5-1.el7.x86_64 3/5 2019-11-25T09:13:06.260 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 4/5 2019-11-25T09:13:06.713 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 5/5 2019-11-25T09:13:06.713 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:06.714 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:13:06.714 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-test.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:06.714 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:06.714 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:13:06.715 INFO:teuthology.orchestra.run.smithi153.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-11-25T09:13:06.715 INFO:teuthology.orchestra.run.smithi153.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-11-25T09:13:06.715 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:06.715 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:13:06.836 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph 2019-11-25T09:13:07.130 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:07.177 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:07.178 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:13:07.178 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:13:07.179 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:07.179 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:07.208 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : jq-1.5-1.el7.x86_64 4/5 2019-11-25T09:13:07.538 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 5/5 2019-11-25T09:13:07.646 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : oniguruma-5.9.5-3.el7.x86_64 1/5 2019-11-25T09:13:07.763 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : socat-1.7.3.2-2.el7.x86_64 2/5 2019-11-25T09:13:07.829 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:07.880 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : jq-1.5-1.el7.x86_64 3/5 2019-11-25T09:13:08.014 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : xmlstarlet-1.6.1-1.el7.x86_64 4/5 2019-11-25T09:13:08.173 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:13:08.173 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:08.173 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:08.175 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.205 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.206 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.206 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.208 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:08.208 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:08.218 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:08.220 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.221 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.223 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.224 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.225 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.226 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:08.227 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:08.230 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:08.238 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:08.238 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-11-25T09:13:08.238 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-11-25T09:13:08.238 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-11-25T09:13:08.239 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:08.239 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:08.240 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:08.240 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:08.241 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:08.241 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-11-25T09:13:08.242 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-11-25T09:13:08.243 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-11-25T09:13:08.243 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-11-25T09:13:08.243 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-11-25T09:13:08.244 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:08.245 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-11-25T09:13:08.245 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-11-25T09:13:08.246 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-11-25T09:13:08.246 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-11-25T09:13:08.246 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-11-25T09:13:08.247 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-11-25T09:13:08.249 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-11-25T09:13:08.249 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-11-25T09:13:08.250 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-11-25T09:13:08.250 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:08.250 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-11-25T09:13:08.250 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-11-25T09:13:08.251 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-11-25T09:13:08.251 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:08.251 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-11-25T09:13:08.251 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-11-25T09:13:08.252 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:08.252 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-11-25T09:13:08.524 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-test-15.0.0-7614.g12b9659.el7.x86_64 5/5 2019-11-25T09:13:08.524 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:08.524 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:13:08.525 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-test.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:08.525 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:08.525 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:13:08.525 INFO:teuthology.orchestra.run.smithi081.stdout: jq.x86_64 0:1.5-1.el7 oniguruma.x86_64 0:5.9.5-3.el7 2019-11-25T09:13:08.525 INFO:teuthology.orchestra.run.smithi081.stdout: socat.x86_64 0:1.7.3.2-2.el7 xmlstarlet.x86_64 0:1.6.1-1.el7 2019-11-25T09:13:08.525 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:08.525 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:13:08.610 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:08.653 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph 2019-11-25T09:13:08.842 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:08.843 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:13:08.845 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:08.845 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:08.847 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository 2019-11-25T09:13:08.847 INFO:teuthology.orchestra.run.smithi153.stdout: Size 2019-11-25T09:13:08.847 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:08.848 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:13:08.848 INFO:teuthology.orchestra.run.smithi153.stdout: ceph x86_64 2:15.0.0-7614.g12b9659.el7 ceph 2.0 k 2019-11-25T09:13:08.848 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:13:08.848 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds x86_64 2:15.0.0-7614.g12b9659.el7 ceph 1.9 M 2019-11-25T09:13:08.848 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr x86_64 2:15.0.0-7614.g12b9659.el7 ceph 1.7 M 2019-11-25T09:13:08.848 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon x86_64 2:15.0.0-7614.g12b9659.el7 ceph 4.3 M 2019-11-25T09:13:08.849 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd x86_64 2:15.0.0-7614.g12b9659.el7 ceph 16 M 2019-11-25T09:13:08.849 INFO:teuthology.orchestra.run.smithi153.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-11-25T09:13:08.849 INFO:teuthology.orchestra.run.smithi153.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-11-25T09:13:08.849 INFO:teuthology.orchestra.run.smithi153.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-11-25T09:13:08.849 INFO:teuthology.orchestra.run.smithi153.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-11-25T09:13:08.849 INFO:teuthology.orchestra.run.smithi153.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-11-25T09:13:08.850 INFO:teuthology.orchestra.run.smithi153.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-11-25T09:13:08.850 INFO:teuthology.orchestra.run.smithi153.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-11-25T09:13:08.850 INFO:teuthology.orchestra.run.smithi153.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-11-25T09:13:08.850 INFO:teuthology.orchestra.run.smithi153.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-11-25T09:13:08.850 INFO:teuthology.orchestra.run.smithi153.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-11-25T09:13:08.850 INFO:teuthology.orchestra.run.smithi153.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-11-25T09:13:08.850 INFO:teuthology.orchestra.run.smithi153.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-11-25T09:13:08.851 INFO:teuthology.orchestra.run.smithi153.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-11-25T09:13:08.851 INFO:teuthology.orchestra.run.smithi153.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-11-25T09:13:08.851 INFO:teuthology.orchestra.run.smithi153.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-11-25T09:13:08.851 INFO:teuthology.orchestra.run.smithi153.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-11-25T09:13:08.851 INFO:teuthology.orchestra.run.smithi153.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-11-25T09:13:08.851 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:08.851 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:13:08.852 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:08.852 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+21 Dependent packages) 2019-11-25T09:13:08.852 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:08.852 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 28 M 2019-11-25T09:13:08.853 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 87 M 2019-11-25T09:13:08.853 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:13:08.891 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:08.930 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:08.931 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:13:08.932 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:13:08.932 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:08.932 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:09.569 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:09.890 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:13:09.890 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:09.890 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:09.892 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: ceph-mon = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.921 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: ceph-mgr = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.922 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: ceph-mds = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.923 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: ceph-osd = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.925 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:09.925 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-mds.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:09.935 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-mgr.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:09.937 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: pyOpenSSL for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.938 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-requests for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.940 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-werkzeug for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.941 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-cherrypy for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.941 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-bcrypt for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.943 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-pecan for package: 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:13:09.943 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-mon.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:09.946 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-osd.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:09.950 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:09.951 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package pyOpenSSL.x86_64 0:0.13.1-4.el7 will be installed 2019-11-25T09:13:09.951 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-cherrypy.noarch 0:3.2.2-4.el7 will be installed 2019-11-25T09:13:09.952 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-pecan.noarch 0:0.4.5-2.el7 will be installed 2019-11-25T09:13:09.953 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-mako >= 0.4.0 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:09.954 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-simplegeneric >= 0.8 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:09.954 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-webob >= 1.2 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:09.955 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-webtest >= 1.3.1 for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:09.956 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-singledispatch for package: python-pecan-0.4.5-2.el7.noarch 2019-11-25T09:13:09.957 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-requests.noarch 0:2.6.0-7.el7_7 will be installed 2019-11-25T09:13:09.957 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-urllib3 >= 1.10.2-1 for package: python-requests-2.6.0-7.el7_7.noarch 2019-11-25T09:13:09.958 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-werkzeug.noarch 0:0.9.1-2.el7 will be installed 2019-11-25T09:13:09.959 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-bcrypt.x86_64 0:3.1.6-2.el7 will be installed 2019-11-25T09:13:09.959 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python2-six for package: python2-bcrypt-3.1.6-2.el7.x86_64 2019-11-25T09:13:09.960 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:09.960 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-mako.noarch 0:0.8.1-2.el7 will be installed 2019-11-25T09:13:09.960 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-markupsafe for package: python-mako-0.8.1-2.el7.noarch 2019-11-25T09:13:09.961 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-beaker for package: python-mako-0.8.1-2.el7.noarch 2019-11-25T09:13:09.962 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-simplegeneric.noarch 0:0.8-7.el7 will be installed 2019-11-25T09:13:09.962 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-singledispatch.noarch 0:3.4.0.2-2.el7 will be installed 2019-11-25T09:13:09.962 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-urllib3.noarch 0:1.10.2-7.el7 will be installed 2019-11-25T09:13:09.963 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-webob.noarch 0:1.2.3-7.el7 will be installed 2019-11-25T09:13:09.963 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-webtest.noarch 0:1.3.4-6.el7 will be installed 2019-11-25T09:13:09.963 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-six.noarch 0:1.9.0-0.el7 will be installed 2019-11-25T09:13:09.963 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-beaker.noarch 0:1.5.4-10.el7 will be installed 2019-11-25T09:13:09.964 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-paste for package: python-beaker-1.5.4-10.el7.noarch 2019-11-25T09:13:09.965 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-markupsafe.x86_64 0:0.11-10.el7 will be installed 2019-11-25T09:13:09.965 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:09.965 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 will be installed 2019-11-25T09:13:09.965 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-tempita for package: python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 2019-11-25T09:13:09.966 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:09.966 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-tempita.noarch 0:0.5.1-6.el7 will be installed 2019-11-25T09:13:10.325 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:10.546 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:10.547 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:13:10.549 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:10.549 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:10.549 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository 2019-11-25T09:13:10.550 INFO:teuthology.orchestra.run.smithi081.stdout: Size 2019-11-25T09:13:10.550 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:10.550 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:13:10.550 INFO:teuthology.orchestra.run.smithi081.stdout: ceph x86_64 2:15.0.0-7614.g12b9659.el7 ceph 2.0 k 2019-11-25T09:13:10.550 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:13:10.550 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds x86_64 2:15.0.0-7614.g12b9659.el7 ceph 1.9 M 2019-11-25T09:13:10.551 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr x86_64 2:15.0.0-7614.g12b9659.el7 ceph 1.7 M 2019-11-25T09:13:10.551 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon x86_64 2:15.0.0-7614.g12b9659.el7 ceph 4.3 M 2019-11-25T09:13:10.551 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd x86_64 2:15.0.0-7614.g12b9659.el7 ceph 16 M 2019-11-25T09:13:10.551 INFO:teuthology.orchestra.run.smithi081.stdout: pyOpenSSL x86_64 0.13.1-4.el7 base 135 k 2019-11-25T09:13:10.551 INFO:teuthology.orchestra.run.smithi081.stdout: python-beaker noarch 1.5.4-10.el7 base 80 k 2019-11-25T09:13:10.551 INFO:teuthology.orchestra.run.smithi081.stdout: python-cherrypy noarch 3.2.2-4.el7 base 422 k 2019-11-25T09:13:10.551 INFO:teuthology.orchestra.run.smithi081.stdout: python-mako noarch 0.8.1-2.el7 base 307 k 2019-11-25T09:13:10.552 INFO:teuthology.orchestra.run.smithi081.stdout: python-markupsafe x86_64 0.11-10.el7 base 25 k 2019-11-25T09:13:10.552 INFO:teuthology.orchestra.run.smithi081.stdout: python-paste noarch 1.7.5.1-9.20111221hg1498.el7 base 866 k 2019-11-25T09:13:10.552 INFO:teuthology.orchestra.run.smithi081.stdout: python-pecan noarch 0.4.5-2.el7 epel 255 k 2019-11-25T09:13:10.552 INFO:teuthology.orchestra.run.smithi081.stdout: python-requests noarch 2.6.0-7.el7_7 updates 95 k 2019-11-25T09:13:10.552 INFO:teuthology.orchestra.run.smithi081.stdout: python-simplegeneric noarch 0.8-7.el7 epel 12 k 2019-11-25T09:13:10.552 INFO:teuthology.orchestra.run.smithi081.stdout: python-singledispatch noarch 3.4.0.2-2.el7 epel 18 k 2019-11-25T09:13:10.552 INFO:teuthology.orchestra.run.smithi081.stdout: python-tempita noarch 0.5.1-6.el7 base 33 k 2019-11-25T09:13:10.553 INFO:teuthology.orchestra.run.smithi081.stdout: python-urllib3 noarch 1.10.2-7.el7 base 103 k 2019-11-25T09:13:10.553 INFO:teuthology.orchestra.run.smithi081.stdout: python-webob noarch 1.2.3-7.el7 base 202 k 2019-11-25T09:13:10.553 INFO:teuthology.orchestra.run.smithi081.stdout: python-webtest noarch 1.3.4-6.el7 base 102 k 2019-11-25T09:13:10.553 INFO:teuthology.orchestra.run.smithi081.stdout: python-werkzeug noarch 0.9.1-2.el7 extras 562 k 2019-11-25T09:13:10.553 INFO:teuthology.orchestra.run.smithi081.stdout: python2-bcrypt x86_64 3.1.6-2.el7 epel 39 k 2019-11-25T09:13:10.553 INFO:teuthology.orchestra.run.smithi081.stdout: python2-six noarch 1.9.0-0.el7 epel 2.9 k 2019-11-25T09:13:10.553 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:10.554 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:13:10.554 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:10.554 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+21 Dependent packages) 2019-11-25T09:13:10.554 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:10.555 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 28 M 2019-11-25T09:13:10.555 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 87 M 2019-11-25T09:13:10.555 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:13:10.587 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:10.587 INFO:teuthology.orchestra.run.smithi153.stdout:Total 16 MB/s | 28 MB 00:01 2019-11-25T09:13:10.606 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:13:10.644 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:13:10.958 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:13:10.958 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:13:11.794 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-11-25T09:13:11.991 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:11.992 INFO:teuthology.orchestra.run.smithi081.stdout:Total 19 MB/s | 28 MB 00:01 2019-11-25T09:13:12.019 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:13:12.067 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:13:12.121 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-11-25T09:13:12.375 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:13:12.375 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:13:12.388 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-11-25T09:13:12.844 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-11-25T09:13:13.107 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-webob-1.2.3-7.el7.noarch 1/22 2019-11-25T09:13:13.210 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-11-25T09:13:13.433 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : pyOpenSSL-0.13.1-4.el7.x86_64 2/22 2019-11-25T09:13:13.636 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-11-25T09:13:13.700 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-webtest-1.3.4-6.el7.noarch 3/22 2019-11-25T09:13:14.155 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-11-25T09:13:14.236 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-tempita-0.5.1-6.el7.noarch 4/22 2019-11-25T09:13:14.497 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-11-25T09:13:14.916 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 9/22 2019-11-25T09:13:15.122 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 5/22 2019-11-25T09:13:15.549 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-beaker-1.5.4-10.el7.noarch 6/22 2019-11-25T09:13:15.603 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-mako-0.8.1-2.el7.noarch 10/22 2019-11-25T09:13:16.062 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-cherrypy-3.2.2-4.el7.noarch 7/22 2019-11-25T09:13:16.141 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-11-25T09:13:16.387 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-werkzeug-0.9.1-2.el7.noarch 8/22 2019-11-25T09:13:16.834 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-markupsafe-0.11-10.el7.x86_64 9/22 2019-11-25T09:13:17.059 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-mds-15.0.0-7614.g12b9659.el7.x86_64 12/22 2019-11-25T09:13:17.150 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-mako-0.8.1-2.el7.noarch 10/22 2019-11-25T09:13:17.543 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-mon-15.0.0-7614.g12b9659.el7.x86_64 13/22 2019-11-25T09:13:17.664 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-11-25T09:13:17.840 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-11-25T09:13:18.573 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-mds-15.0.0-7614.g12b9659.el7.x86_64 12/22 2019-11-25T09:13:19.030 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-mon-15.0.0-7614.g12b9659.el7.x86_64 13/22 2019-11-25T09:13:19.396 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-urllib3-1.10.2-7.el7.noarch 14/22 2019-11-25T09:13:19.702 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-11-25T09:13:20.103 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-osd-15.0.0-7614.g12b9659.el7.x86_64 16/22 2019-11-25T09:13:20.357 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-11-25T09:13:20.663 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-11-25T09:13:21.081 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-11-25T09:13:21.269 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-requests-2.6.0-7.el7_7.noarch 15/22 2019-11-25T09:13:21.652 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-osd-15.0.0-7614.g12b9659.el7.x86_64 16/22 2019-11-25T09:13:21.751 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-11-25T09:13:21.871 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python2-six-1.9.0-0.el7.noarch 17/22 2019-11-25T09:13:22.161 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python2-bcrypt-3.1.6-2.el7.x86_64 18/22 2019-11-25T09:13:22.169 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 21/22 2019-11-25T09:13:22.472 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 22/22 2019-11-25T09:13:22.553 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-simplegeneric-0.8-7.el7.noarch 19/22 2019-11-25T09:13:22.623 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-11-25T09:13:22.727 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-11-25T09:13:22.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-11-25T09:13:23.011 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-osd-15.0.0-7614.g12b9659.el7.x86_64 4/22 2019-11-25T09:13:23.144 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 5/22 2019-11-25T09:13:23.286 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 6/22 2019-11-25T09:13:23.303 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-pecan-0.4.5-2.el7.noarch 20/22 2019-11-25T09:13:23.387 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-mon-15.0.0-7614.g12b9659.el7.x86_64 7/22 2019-11-25T09:13:23.495 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 8/22 2019-11-25T09:13:23.638 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 9/22 2019-11-25T09:13:23.675 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 21/22 2019-11-25T09:13:23.763 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-mds-15.0.0-7614.g12b9659.el7.x86_64 10/22 2019-11-25T09:13:23.897 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-11-25T09:13:23.996 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 22/22 2019-11-25T09:13:23.997 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-11-25T09:13:24.073 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 13/22 2019-11-25T09:13:24.082 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-simplegeneric-0.8-7.el7.noarch 1/22 2019-11-25T09:13:24.165 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 14/22 2019-11-25T09:13:24.165 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python2-six-1.9.0-0.el7.noarch 2/22 2019-11-25T09:13:24.285 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 15/22 2019-11-25T09:13:24.374 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-11-25T09:13:24.458 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : pyOpenSSL-0.13.1-4.el7.x86_64 3/22 2019-11-25T09:13:24.491 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-11-25T09:13:24.575 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 18/22 2019-11-25T09:13:24.618 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-osd-15.0.0-7614.g12b9659.el7.x86_64 4/22 2019-11-25T09:13:24.743 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-urllib3-1.10.2-7.el7.noarch 5/22 2019-11-25T09:13:24.759 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 19/22 2019-11-25T09:13:24.843 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-mako-0.8.1-2.el7.noarch 6/22 2019-11-25T09:13:24.868 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 20/22 2019-11-25T09:13:24.952 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-mon-15.0.0-7614.g12b9659.el7.x86_64 7/22 2019-11-25T09:13:25.026 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 21/22 2019-11-25T09:13:25.061 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-requests-2.6.0-7.el7_7.noarch 8/22 2019-11-25T09:13:25.203 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-pecan-0.4.5-2.el7.noarch 9/22 2019-11-25T09:13:25.337 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-mds-15.0.0-7614.g12b9659.el7.x86_64 10/22 2019-11-25T09:13:25.462 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-singledispatch-3.4.0.2-2.el7.noarch 11/22 2019-11-25T09:13:25.563 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-markupsafe-0.11-10.el7.x86_64 12/22 2019-11-25T09:13:25.631 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-11-25T09:13:25.631 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:25.631 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:13:25.631 INFO:teuthology.orchestra.run.smithi153.stdout: ceph.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:25.631 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:25.632 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:13:25.632 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mds.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:25.632 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:25.632 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mon.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:25.632 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-osd.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:25.633 INFO:teuthology.orchestra.run.smithi153.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-11-25T09:13:25.633 INFO:teuthology.orchestra.run.smithi153.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-11-25T09:13:25.633 INFO:teuthology.orchestra.run.smithi153.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-11-25T09:13:25.633 INFO:teuthology.orchestra.run.smithi153.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-11-25T09:13:25.633 INFO:teuthology.orchestra.run.smithi153.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-11-25T09:13:25.634 INFO:teuthology.orchestra.run.smithi153.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-11-25T09:13:25.634 INFO:teuthology.orchestra.run.smithi153.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-11-25T09:13:25.634 INFO:teuthology.orchestra.run.smithi153.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-11-25T09:13:25.634 INFO:teuthology.orchestra.run.smithi153.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-11-25T09:13:25.635 INFO:teuthology.orchestra.run.smithi153.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-11-25T09:13:25.635 INFO:teuthology.orchestra.run.smithi153.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-11-25T09:13:25.635 INFO:teuthology.orchestra.run.smithi153.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-11-25T09:13:25.635 INFO:teuthology.orchestra.run.smithi153.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-11-25T09:13:25.635 INFO:teuthology.orchestra.run.smithi153.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-11-25T09:13:25.636 INFO:teuthology.orchestra.run.smithi153.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-11-25T09:13:25.636 INFO:teuthology.orchestra.run.smithi153.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-11-25T09:13:25.636 INFO:teuthology.orchestra.run.smithi153.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-11-25T09:13:25.636 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:25.637 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:13:25.646 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-15.0.0-7614.g12b9659.el7.x86_64 13/22 2019-11-25T09:13:25.755 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-paste-1.7.5.1-9.20111221hg1498.el7.noarch 14/22 2019-11-25T09:13:25.761 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-daemon 2019-11-25T09:13:25.889 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 15/22 2019-11-25T09:13:25.990 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-webob-1.2.3-7.el7.noarch 16/22 2019-11-25T09:13:26.055 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:26.101 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-werkzeug-0.9.1-2.el7.noarch 17/22 2019-11-25T09:13:26.109 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:26.109 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:13:26.109 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:13:26.109 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:26.109 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:26.182 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-cherrypy-3.2.2-4.el7.noarch 18/22 2019-11-25T09:13:26.308 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-beaker-1.5.4-10.el7.noarch 19/22 2019-11-25T09:13:26.409 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-tempita-0.5.1-6.el7.noarch 20/22 2019-11-25T09:13:26.567 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python2-bcrypt-3.1.6-2.el7.x86_64 21/22 2019-11-25T09:13:26.750 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:27.045 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-webtest-1.3.4-6.el7.noarch 22/22 2019-11-25T09:13:27.045 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:13:27.046 INFO:teuthology.orchestra.run.smithi081.stdout: ceph.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:27.046 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:27.046 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:13:27.046 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mds.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:27.046 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:27.046 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mon.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:27.047 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-osd.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:27.047 INFO:teuthology.orchestra.run.smithi081.stdout: pyOpenSSL.x86_64 0:0.13.1-4.el7 2019-11-25T09:13:27.047 INFO:teuthology.orchestra.run.smithi081.stdout: python-beaker.noarch 0:1.5.4-10.el7 2019-11-25T09:13:27.047 INFO:teuthology.orchestra.run.smithi081.stdout: python-cherrypy.noarch 0:3.2.2-4.el7 2019-11-25T09:13:27.047 INFO:teuthology.orchestra.run.smithi081.stdout: python-mako.noarch 0:0.8.1-2.el7 2019-11-25T09:13:27.047 INFO:teuthology.orchestra.run.smithi081.stdout: python-markupsafe.x86_64 0:0.11-10.el7 2019-11-25T09:13:27.047 INFO:teuthology.orchestra.run.smithi081.stdout: python-paste.noarch 0:1.7.5.1-9.20111221hg1498.el7 2019-11-25T09:13:27.048 INFO:teuthology.orchestra.run.smithi081.stdout: python-pecan.noarch 0:0.4.5-2.el7 2019-11-25T09:13:27.048 INFO:teuthology.orchestra.run.smithi081.stdout: python-requests.noarch 0:2.6.0-7.el7_7 2019-11-25T09:13:27.048 INFO:teuthology.orchestra.run.smithi081.stdout: python-simplegeneric.noarch 0:0.8-7.el7 2019-11-25T09:13:27.048 INFO:teuthology.orchestra.run.smithi081.stdout: python-singledispatch.noarch 0:3.4.0.2-2.el7 2019-11-25T09:13:27.048 INFO:teuthology.orchestra.run.smithi081.stdout: python-tempita.noarch 0:0.5.1-6.el7 2019-11-25T09:13:27.048 INFO:teuthology.orchestra.run.smithi081.stdout: python-urllib3.noarch 0:1.10.2-7.el7 2019-11-25T09:13:27.048 INFO:teuthology.orchestra.run.smithi081.stdout: python-webob.noarch 0:1.2.3-7.el7 2019-11-25T09:13:27.049 INFO:teuthology.orchestra.run.smithi081.stdout: python-webtest.noarch 0:1.3.4-6.el7 2019-11-25T09:13:27.049 INFO:teuthology.orchestra.run.smithi081.stdout: python-werkzeug.noarch 0:0.9.1-2.el7 2019-11-25T09:13:27.049 INFO:teuthology.orchestra.run.smithi081.stdout: python2-bcrypt.x86_64 0:3.1.6-2.el7 2019-11-25T09:13:27.049 INFO:teuthology.orchestra.run.smithi081.stdout: python2-six.noarch 0:1.9.0-0.el7 2019-11-25T09:13:27.049 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:27.049 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:13:27.095 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:13:27.096 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:27.096 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:27.179 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-daemon 2019-11-25T09:13:27.409 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:27.459 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:27.459 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:13:27.459 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:13:27.459 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:27.461 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:27.461 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:27.687 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:27.688 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:13:27.688 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:27.689 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository 2019-11-25T09:13:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: Size 2019-11-25T09:13:27.689 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:27.689 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:13:27.689 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-daemon x86_64 2:15.0.0-7614.g12b9659.el7 ceph 17 k 2019-11-25T09:13:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:27.690 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:13:27.690 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:27.690 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2019-11-25T09:13:27.690 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:27.690 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 17 k 2019-11-25T09:13:27.691 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 66 k 2019-11-25T09:13:27.691 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:13:28.027 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:13:28.038 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:13:28.049 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:13:28.049 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:13:28.087 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:28.406 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:13:28.407 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:28.407 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-daemon.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:28.773 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:28.955 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-daemon-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:13:28.996 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:28.996 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:13:28.997 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:28.997 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:28.997 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository 2019-11-25T09:13:28.998 INFO:teuthology.orchestra.run.smithi081.stdout: Size 2019-11-25T09:13:28.998 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:28.998 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:13:28.998 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-daemon x86_64 2:15.0.0-7614.g12b9659.el7 ceph 17 k 2019-11-25T09:13:28.998 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:28.998 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:13:28.998 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:28.999 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2019-11-25T09:13:28.999 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:28.999 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 17 k 2019-11-25T09:13:28.999 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 66 k 2019-11-25T09:13:28.999 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:13:29.311 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:13:29.319 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:13:29.321 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:13:29.322 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:13:29.442 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-daemon-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:13:29.442 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:29.443 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:13:29.443 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-daemon.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:29.443 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:29.443 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:13:29.579 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr 2019-11-25T09:13:29.825 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:29.871 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:29.871 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:13:29.871 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:13:29.872 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:29.872 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:30.152 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-daemon-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:13:30.521 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:30.617 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-daemon-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:13:30.617 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:30.618 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:13:30.618 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-daemon.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:30.618 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:30.618 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:13:30.739 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr 2019-11-25T09:13:30.855 INFO:teuthology.orchestra.run.smithi153.stdout:Package 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 already installed and latest version 2019-11-25T09:13:30.856 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do 2019-11-25T09:13:30.919 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-dashboard 2019-11-25T09:13:31.028 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:31.073 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:31.075 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:13:31.075 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:13:31.076 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:31.076 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:31.203 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:31.248 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:31.249 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:13:31.251 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:13:31.255 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:31.255 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:31.712 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:31.900 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:32.053 INFO:teuthology.orchestra.run.smithi081.stdout:Package 2:ceph-mgr-15.0.0-7614.g12b9659.el7.x86_64 already installed and latest version 2019-11-25T09:13:32.053 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do 2019-11-25T09:13:32.113 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-dashboard 2019-11-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:13:32.236 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:32.237 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:32.239 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:13:32.269 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:32.270 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-11-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-11-25T09:13:32.271 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:32.272 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-11-25T09:13:32.362 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:32.407 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:32.408 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:13:32.409 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:13:32.409 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:32.409 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:32.635 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:32.865 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:32.865 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:13:32.865 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:32.866 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:32.866 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T09:13:32.866 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:32.866 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:13:32.866 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 3.8 M 2019-11-25T09:13:32.867 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:13:32.867 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 18 k 2019-11-25T09:13:32.867 INFO:teuthology.orchestra.run.smithi153.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-11-25T09:13:32.867 INFO:teuthology.orchestra.run.smithi153.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-11-25T09:13:32.868 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:32.868 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:13:32.868 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:32.868 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+3 Dependent packages) 2019-11-25T09:13:32.868 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:32.868 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 4.5 M 2019-11-25T09:13:32.868 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 19 M 2019-11-25T09:13:32.868 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:13:33.057 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:33.399 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:13:33.400 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:33.400 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-mgr-dashboard.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:33.400 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: ceph-grafana-dashboards = 2:15.0.0-7614.g12b9659.el7 for package: 2:ceph-mgr-dashboard-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:13:33.426 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-routes for package: 2:ceph-mgr-dashboard-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:13:33.426 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:33.427 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-grafana-dashboards.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:33.427 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-routes.noarch 0:1.13-2.el7 will be installed 2019-11-25T09:13:33.428 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-repoze-lru for package: python-routes-1.13-2.el7.noarch 2019-11-25T09:13:33.428 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:33.429 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-repoze-lru.noarch 0:0.4-3.el7 will be installed 2019-11-25T09:13:33.484 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:33.484 INFO:teuthology.orchestra.run.smithi153.stdout:Total 7.3 MB/s | 4.5 MB 00:00 2019-11-25T09:13:33.490 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:13:33.500 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:13:33.675 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:13:33.675 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:13:33.791 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:34.014 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:34.014 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:13:34.016 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:34.016 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:34.016 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2019-11-25T09:13:34.016 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:34.016 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:13:34.017 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 3.8 M 2019-11-25T09:13:34.017 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:13:34.017 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 18 k 2019-11-25T09:13:34.017 INFO:teuthology.orchestra.run.smithi081.stdout: python-repoze-lru noarch 0.4-3.el7 epel 13 k 2019-11-25T09:13:34.017 INFO:teuthology.orchestra.run.smithi081.stdout: python-routes noarch 1.13-2.el7 epel 640 k 2019-11-25T09:13:34.017 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:34.017 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:13:34.018 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:34.018 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+3 Dependent packages) 2019-11-25T09:13:34.018 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:34.018 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 4.5 M 2019-11-25T09:13:34.018 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 19 M 2019-11-25T09:13:34.018 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:13:34.376 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7614.g12b9659.el7.noarch 1/4 2019-11-25T09:13:34.697 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:34.697 INFO:teuthology.orchestra.run.smithi081.stdout:Total 6.6 MB/s | 4.5 MB 00:00 2019-11-25T09:13:34.704 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:13:34.715 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:13:34.809 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-11-25T09:13:34.895 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:13:34.895 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:13:35.601 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-grafana-dashboards-15.0.0-7614.g12b9659.el7.noarch 1/4 2019-11-25T09:13:35.977 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-11-25T09:13:36.044 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-repoze-lru-0.4-3.el7.noarch 2/4 2019-11-25T09:13:36.594 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7614.g12b9659.el7.noarch 4/4 2019-11-25T09:13:36.679 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-11-25T09:13:36.796 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-routes-1.13-2.el7.noarch 2/4 2019-11-25T09:13:36.922 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7614.g12b9659.el7.noarch 3/4 2019-11-25T09:13:37.214 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-routes-1.13-2.el7.noarch 3/4 2019-11-25T09:13:37.383 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7614.g12b9659.el7.noarch 4/4 2019-11-25T09:13:37.384 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:37.384 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:13:37.384 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:37.384 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:37.385 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:13:37.385 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:37.385 INFO:teuthology.orchestra.run.smithi153.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-11-25T09:13:37.385 INFO:teuthology.orchestra.run.smithi153.stdout: python-routes.noarch 0:1.13-2.el7 2019-11-25T09:13:37.386 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:37.386 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:13:37.515 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-11-25T09:13:37.629 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-mgr-dashboard-15.0.0-7614.g12b9659.el7.noarch 4/4 2019-11-25T09:13:37.730 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-repoze-lru-0.4-3.el7.noarch 1/4 2019-11-25T09:13:37.788 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:37.835 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:37.835 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:13:37.836 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:13:37.837 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:37.837 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:37.847 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-routes-1.13-2.el7.noarch 2/4 2019-11-25T09:13:37.947 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-mgr-dashboard-15.0.0-7614.g12b9659.el7.noarch 3/4 2019-11-25T09:13:38.416 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-grafana-dashboards-15.0.0-7614.g12b9659.el7.noarch 4/4 2019-11-25T09:13:38.416 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:38.417 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:13:38.417 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-dashboard.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:38.417 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:38.417 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:13:38.417 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-grafana-dashboards.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:38.417 INFO:teuthology.orchestra.run.smithi081.stdout: python-repoze-lru.noarch 0:0.4-3.el7 2019-11-25T09:13:38.417 INFO:teuthology.orchestra.run.smithi081.stdout: python-routes.noarch 0:1.13-2.el7 2019-11-25T09:13:38.418 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:38.418 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:13:38.499 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:38.544 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-diskprediction-cloud 2019-11-25T09:13:38.776 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:38.820 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:13:38.821 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:13:38.822 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:38.826 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:38.836 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:13:38.836 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:38.837 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:39.202 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:39.428 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:39.429 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:13:39.429 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:39.430 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:39.430 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T09:13:39.430 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:39.430 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:13:39.430 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-cloud 2019-11-25T09:13:39.430 INFO:teuthology.orchestra.run.smithi153.stdout: noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 82 k 2019-11-25T09:13:39.430 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:39.431 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:13:39.431 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:39.431 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2019-11-25T09:13:39.431 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:39.431 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 82 k 2019-11-25T09:13:39.431 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 411 k 2019-11-25T09:13:39.432 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:13:39.470 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:39.769 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:13:39.772 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:13:39.803 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:13:39.804 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:39.804 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:39.827 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:13:39.828 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:13:40.170 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:40.395 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:40.395 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:13:40.395 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:40.395 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:40.395 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2019-11-25T09:13:40.395 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:40.396 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:13:40.396 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-cloud 2019-11-25T09:13:40.396 INFO:teuthology.orchestra.run.smithi081.stdout: noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 82 k 2019-11-25T09:13:40.396 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:40.396 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:13:40.396 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:40.396 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2019-11-25T09:13:40.397 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:40.397 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 82 k 2019-11-25T09:13:40.397 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 411 k 2019-11-25T09:13:40.397 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:13:40.561 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7614.g12b9659.el7.n 1/1 2019-11-25T09:13:40.728 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:13:40.736 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:13:40.783 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:13:40.784 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:13:41.134 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7614.g12b9659.el7.n 1/1 2019-11-25T09:13:41.134 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:41.135 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:13:41.135 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:41.135 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:41.135 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:13:41.245 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-diskprediction-local 2019-11-25T09:13:41.456 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-mgr-diskprediction-cloud-15.0.0-7614.g12b9659.el7.n 1/1 2019-11-25T09:13:41.462 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:41.500 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:41.501 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:41.502 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:41.903 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-mgr-diskprediction-cloud-15.0.0-7614.g12b9659.el7.n 1/1 2019-11-25T09:13:41.903 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:41.903 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:13:41.904 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-cloud.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:41.904 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:41.904 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:13:42.017 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-diskprediction-local 2019-11-25T09:13:42.138 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:42.276 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:42.321 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:42.322 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:13:42.322 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:13:42.323 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:42.323 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:42.463 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:13:42.463 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:42.463 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:42.469 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:13:42.497 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:42.497 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-11-25T09:13:42.502 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-11-25T09:13:42.503 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-11-25T09:13:42.505 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-11-25T09:13:42.506 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:42.506 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-11-25T09:13:42.506 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-11-25T09:13:42.507 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-11-25T09:13:42.509 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-11-25T09:13:42.510 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:42.510 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-11-25T09:13:42.872 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:42.968 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:43.101 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:43.102 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:13:43.103 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:43.103 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:43.103 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T09:13:43.104 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:43.104 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:13:43.104 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local 2019-11-25T09:13:43.104 INFO:teuthology.orchestra.run.smithi153.stdout: noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 8.6 M 2019-11-25T09:13:43.104 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:13:43.104 INFO:teuthology.orchestra.run.smithi153.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-11-25T09:13:43.104 INFO:teuthology.orchestra.run.smithi153.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-11-25T09:13:43.105 INFO:teuthology.orchestra.run.smithi153.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-11-25T09:13:43.105 INFO:teuthology.orchestra.run.smithi153.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-11-25T09:13:43.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:43.105 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:13:43.105 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:43.105 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+4 Dependent packages) 2019-11-25T09:13:43.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:43.106 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 19 M 2019-11-25T09:13:43.106 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 104 M 2019-11-25T09:13:43.106 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:13:43.299 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:13:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:43.300 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-mgr-diskprediction-local.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:43.302 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: scipy for package: 2:ceph-mgr-diskprediction-local-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:13:43.331 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:43.332 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package scipy.x86_64 0:0.12.1-6.el7 will be installed 2019-11-25T09:13:43.338 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: f2py for package: scipy-0.12.1-6.el7.x86_64 2019-11-25T09:13:43.339 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libumfpack.so.5()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-11-25T09:13:43.341 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libamd.so.2()(64bit) for package: scipy-0.12.1-6.el7.x86_64 2019-11-25T09:13:43.342 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:43.342 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package numpy-f2py.x86_64 1:1.7.1-13.el7 will be installed 2019-11-25T09:13:43.342 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package suitesparse.x86_64 0:4.0.2-10.el7 will be installed 2019-11-25T09:13:43.343 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libtbbmalloc.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-11-25T09:13:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libtbb.so.2()(64bit) for package: suitesparse-4.0.2-10.el7.x86_64 2019-11-25T09:13:43.345 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:43.346 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package tbb.x86_64 0:4.1-9.20130314.el7 will be installed 2019-11-25T09:13:43.709 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:43.823 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:43.823 INFO:teuthology.orchestra.run.smithi153.stdout:Total 27 MB/s | 19 MB 00:00 2019-11-25T09:13:43.833 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:13:43.858 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:13:43.933 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:43.933 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:13:43.934 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:43.934 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:43.935 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2019-11-25T09:13:43.935 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:43.935 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:13:43.935 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local 2019-11-25T09:13:43.935 INFO:teuthology.orchestra.run.smithi081.stdout: noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 8.6 M 2019-11-25T09:13:43.935 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:13:43.936 INFO:teuthology.orchestra.run.smithi081.stdout: numpy-f2py x86_64 1:1.7.1-13.el7 base 206 k 2019-11-25T09:13:43.936 INFO:teuthology.orchestra.run.smithi081.stdout: scipy x86_64 0.12.1-6.el7 base 9.3 M 2019-11-25T09:13:43.936 INFO:teuthology.orchestra.run.smithi081.stdout: suitesparse x86_64 4.0.2-10.el7 base 928 k 2019-11-25T09:13:43.936 INFO:teuthology.orchestra.run.smithi081.stdout: tbb x86_64 4.1-9.20130314.el7 base 124 k 2019-11-25T09:13:43.936 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:43.936 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:13:43.936 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:43.937 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+4 Dependent packages) 2019-11-25T09:13:43.937 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:43.937 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 19 M 2019-11-25T09:13:43.937 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 104 M 2019-11-25T09:13:43.937 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:13:44.003 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:13:44.004 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:13:44.629 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:44.629 INFO:teuthology.orchestra.run.smithi081.stdout:Total 28 MB/s | 19 MB 00:00 2019-11-25T09:13:44.638 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:13:44.660 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:13:44.763 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:13:44.763 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:13:44.861 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-11-25T09:13:45.360 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-11-25T09:13:45.705 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : tbb-4.1-9.20130314.el7.x86_64 1/5 2019-11-25T09:13:46.172 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : suitesparse-4.0.2-10.el7.x86_64 2/5 2019-11-25T09:13:47.573 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-11-25T09:13:47.921 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 1:numpy-f2py-1.7.1-13.el7.x86_64 3/5 2019-11-25T09:13:49.273 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-11-25T09:13:49.566 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : scipy-0.12.1-6.el7.x86_64 4/5 2019-11-25T09:13:49.657 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7614.g12b9659.el7.n 5/5 2019-11-25T09:13:49.757 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-11-25T09:13:49.900 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 2/5 2019-11-25T09:13:50.008 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7614.g12b9659.el7.n 3/5 2019-11-25T09:13:50.125 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-11-25T09:13:50.252 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-mgr-diskprediction-local-15.0.0-7614.g12b9659.el7.n 5/5 2019-11-25T09:13:50.337 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : suitesparse-4.0.2-10.el7.x86_64 1/5 2019-11-25T09:13:50.438 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : scipy-0.12.1-6.el7.x86_64 2/5 2019-11-25T09:13:50.530 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-mgr-diskprediction-local-15.0.0-7614.g12b9659.el7.n 3/5 2019-11-25T09:13:50.646 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-11-25T09:13:50.646 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:50.646 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:13:50.646 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:50.647 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:50.647 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:13:50.647 INFO:teuthology.orchestra.run.smithi153.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-11-25T09:13:50.647 INFO:teuthology.orchestra.run.smithi153.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-11-25T09:13:50.647 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:50.647 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:13:50.655 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 1:numpy-f2py-1.7.1-13.el7.x86_64 4/5 2019-11-25T09:13:50.779 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-rook 2019-11-25T09:13:51.034 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:51.080 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:51.081 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:13:51.082 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:13:51.082 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:51.082 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:51.124 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : tbb-4.1-9.20130314.el7.x86_64 5/5 2019-11-25T09:13:51.124 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:51.124 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:13:51.124 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-diskprediction-local.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:13:51.124 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:51.124 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:13:51.125 INFO:teuthology.orchestra.run.smithi081.stdout: numpy-f2py.x86_64 1:1.7.1-13.el7 scipy.x86_64 0:0.12.1-6.el7 2019-11-25T09:13:51.125 INFO:teuthology.orchestra.run.smithi081.stdout: suitesparse.x86_64 0:4.0.2-10.el7 tbb.x86_64 0:4.1-9.20130314.el7 2019-11-25T09:13:51.125 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:51.125 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:13:51.240 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-rook 2019-11-25T09:13:51.458 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:13:51.500 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:13:51.501 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:13:51.501 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:13:51.502 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:13:51.502 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:13:51.734 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:52.072 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:13:52.072 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:52.072 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:52.074 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:13:52.082 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:13:52.104 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:52.104 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-11-25T09:13:52.106 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-25T09:13:52.107 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-25T09:13:52.107 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-25T09:13:52.108 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-25T09:13:52.109 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:52.110 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-11-25T09:13:52.110 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-11-25T09:13:52.110 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-11-25T09:13:52.111 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-25T09:13:52.112 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-25T09:13:52.112 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-25T09:13:52.113 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-11-25T09:13:52.114 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-11-25T09:13:52.114 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:52.115 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-11-25T09:13:52.115 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-11-25T09:13:52.115 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-11-25T09:13:52.116 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-11-25T09:13:52.116 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-11-25T09:13:52.117 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:13:52.117 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-11-25T09:13:52.402 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:13:52.402 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:52.402 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-mgr-rook.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:13:52.403 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-kubernetes for package: 2:ceph-mgr-rook-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:13:52.433 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:52.433 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-kubernetes.noarch 0:10.0.1-1.el7 will be installed 2019-11-25T09:13:52.435 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-certifi for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-25T09:13:52.436 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-google-auth for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-25T09:13:52.437 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-requests-oauthlib for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-25T09:13:52.438 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-websocket-client for package: python2-kubernetes-10.0.1-1.el7.noarch 2019-11-25T09:13:52.447 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:52.448 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 will be installed 2019-11-25T09:13:52.448 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-certifi.noarch 0:2018.10.15-5.el7 will be installed 2019-11-25T09:13:52.448 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-google-auth.noarch 1:1.1.1-5.el7 will be installed 2019-11-25T09:13:52.448 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-cachetools for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-25T09:13:52.448 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python2-pyasn1-modules for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-25T09:13:52.448 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python2-rsa for package: 1:python2-google-auth-1.1.1-5.el7.noarch 2019-11-25T09:13:52.449 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-requests-oauthlib.noarch 0:0.8.0-5.el7 will be installed 2019-11-25T09:13:52.449 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python2-oauthlib for package: python2-requests-oauthlib-0.8.0-5.el7.noarch 2019-11-25T09:13:52.449 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:52.449 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-cachetools.noarch 0:1.0.3-1.el7 will be installed 2019-11-25T09:13:52.450 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-oauthlib.noarch 0:2.0.1-8.el7 will be installed 2019-11-25T09:13:52.450 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-jwcrypto for package: python2-oauthlib-2.0.1-8.el7.noarch 2019-11-25T09:13:52.450 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-pyasn1-modules.noarch 0:0.1.9-7.el7 will be installed 2019-11-25T09:13:52.450 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python2-rsa.noarch 0:3.4.1-1.el7 will be installed 2019-11-25T09:13:52.450 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:13:52.451 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-jwcrypto.noarch 0:0.4.2-1.el7 will be installed 2019-11-25T09:13:52.481 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:52.707 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:52.707 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:13:52.708 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:52.709 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:52.709 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T09:13:52.709 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:52.709 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:13:52.709 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 24 k 2019-11-25T09:13:52.710 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:13:52.710 INFO:teuthology.orchestra.run.smithi153.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-11-25T09:13:52.710 INFO:teuthology.orchestra.run.smithi153.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-11-25T09:13:52.710 INFO:teuthology.orchestra.run.smithi153.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-11-25T09:13:52.711 INFO:teuthology.orchestra.run.smithi153.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-11-25T09:13:52.711 INFO:teuthology.orchestra.run.smithi153.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-11-25T09:13:52.711 INFO:teuthology.orchestra.run.smithi153.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-11-25T09:13:52.711 INFO:teuthology.orchestra.run.smithi153.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-11-25T09:13:52.712 INFO:teuthology.orchestra.run.smithi153.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-11-25T09:13:52.712 INFO:teuthology.orchestra.run.smithi153.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-11-25T09:13:52.712 INFO:teuthology.orchestra.run.smithi153.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-11-25T09:13:52.714 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:52.714 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:13:52.714 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:13:52.714 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+10 Dependent packages) 2019-11-25T09:13:52.715 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:13:52.716 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 1.6 M 2019-11-25T09:13:52.716 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 24 M 2019-11-25T09:13:52.716 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:13:52.810 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:13:53.029 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:53.030 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:13:53.031 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:53.031 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:53.031 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2019-11-25T09:13:53.031 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:53.032 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:13:53.032 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 24 k 2019-11-25T09:13:53.032 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:13:53.032 INFO:teuthology.orchestra.run.smithi081.stdout: python-cachetools noarch 1.0.3-1.el7 epel 22 k 2019-11-25T09:13:53.032 INFO:teuthology.orchestra.run.smithi081.stdout: python-jwcrypto noarch 0.4.2-1.el7 base 57 k 2019-11-25T09:13:53.032 INFO:teuthology.orchestra.run.smithi081.stdout: python-websocket-client noarch 0.56.0-3.git3c25814.el7 extras 66 k 2019-11-25T09:13:53.033 INFO:teuthology.orchestra.run.smithi081.stdout: python2-certifi noarch 2018.10.15-5.el7 epel 12 k 2019-11-25T09:13:53.033 INFO:teuthology.orchestra.run.smithi081.stdout: python2-google-auth noarch 1:1.1.1-5.el7 epel 76 k 2019-11-25T09:13:53.033 INFO:teuthology.orchestra.run.smithi081.stdout: python2-kubernetes noarch 10.0.1-1.el7 epel 1.1 M 2019-11-25T09:13:53.033 INFO:teuthology.orchestra.run.smithi081.stdout: python2-oauthlib noarch 2.0.1-8.el7 base 146 k 2019-11-25T09:13:53.033 INFO:teuthology.orchestra.run.smithi081.stdout: python2-pyasn1-modules noarch 0.1.9-7.el7 base 59 k 2019-11-25T09:13:53.033 INFO:teuthology.orchestra.run.smithi081.stdout: python2-requests-oauthlib noarch 0.8.0-5.el7 base 35 k 2019-11-25T09:13:53.033 INFO:teuthology.orchestra.run.smithi081.stdout: python2-rsa noarch 3.4.1-1.el7 epel 67 k 2019-11-25T09:13:53.034 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:53.034 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:13:53.034 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:13:53.034 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+10 Dependent packages) 2019-11-25T09:13:53.034 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:13:53.035 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 1.6 M 2019-11-25T09:13:53.035 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 24 M 2019-11-25T09:13:53.035 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:13:53.832 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:53.832 INFO:teuthology.orchestra.run.smithi081.stdout:Total 2.1 MB/s | 1.6 MB 00:00 2019-11-25T09:13:53.847 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:13:53.860 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:13:53.936 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:13:53.937 INFO:teuthology.orchestra.run.smithi153.stdout:Total 1.3 MB/s | 1.6 MB 00:01 2019-11-25T09:13:53.951 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:13:53.965 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:13:53.965 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:13:53.965 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:13:54.054 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:13:54.054 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:13:54.699 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 1/11 2019-11-25T09:13:54.821 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python2-pyasn1-modules-0.1.9-7.el7.noarch 1/11 2019-11-25T09:13:55.161 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 2/11 2019-11-25T09:13:55.469 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 2/11 2019-11-25T09:13:55.557 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 3/11 2019-11-25T09:13:55.844 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-jwcrypto-0.4.2-1.el7.noarch 3/11 2019-11-25T09:13:56.083 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-11-25T09:13:56.193 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-11-25T09:13:56.420 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 5/11 2019-11-25T09:13:56.499 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python2-requests-oauthlib-0.8.0-5.el7.noarch 5/11 2019-11-25T09:13:56.804 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 6/11 2019-11-25T09:13:56.895 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python2-certifi-2018.10.15-5.el7.noarch 6/11 2019-11-25T09:13:57.174 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-11-25T09:13:57.180 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python2-rsa-3.4.1-1.el7.noarch 7/11 2019-11-25T09:13:57.539 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 8/11 2019-11-25T09:13:57.552 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-cachetools-1.0.3-1.el7.noarch 8/11 2019-11-25T09:13:58.793 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-11-25T09:13:58.823 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 1:python2-google-auth-1.1.1-5.el7.noarch 9/11 2019-11-25T09:13:59.157 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-11-25T09:13:59.220 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python2-kubernetes-10.0.1-1.el7.noarch 10/11 2019-11-25T09:13:59.477 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7614.g12b9659.el7.noarch 11/11 2019-11-25T09:13:59.528 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-mgr-rook-15.0.0-7614.g12b9659.el7.noarch 11/11 2019-11-25T09:13:59.567 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-11-25T09:13:59.623 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python2-requests-oauthlib-0.8.0-5.el7.noarch 1/11 2019-11-25T09:13:59.676 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 2/11 2019-11-25T09:13:59.760 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-11-25T09:13:59.765 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-cachetools-1.0.3-1.el7.noarch 2/11 2019-11-25T09:13:59.866 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python2-rsa-3.4.1-1.el7.noarch 3/11 2019-11-25T09:13:59.903 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-11-25T09:13:59.991 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python2-oauthlib-2.0.1-8.el7.noarch 4/11 2019-11-25T09:14:00.006 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7614.g12b9659.el7.noarch 5/11 2019-11-25T09:14:00.105 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-11-25T09:14:00.108 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-mgr-rook-15.0.0-7614.g12b9659.el7.noarch 5/11 2019-11-25T09:14:00.185 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 1:python2-google-auth-1.1.1-5.el7.noarch 6/11 2019-11-25T09:14:00.204 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-11-25T09:14:00.251 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python2-certifi-2018.10.15-5.el7.noarch 7/11 2019-11-25T09:14:00.333 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-11-25T09:14:00.343 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python2-kubernetes-10.0.1-1.el7.noarch 8/11 2019-11-25T09:14:00.447 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-11-25T09:14:00.469 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-jwcrypto-0.4.2-1.el7.noarch 9/11 2019-11-25T09:14:00.569 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 10/11 2019-11-25T09:14:00.589 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-websocket-client-0.56.0-3.git3c25814.el7.noarch 10/11 2019-11-25T09:14:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 11/11 2019-11-25T09:14:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:01.066 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:14:01.066 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-rook.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:01.067 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:14:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-11-25T09:14:01.067 INFO:teuthology.orchestra.run.smithi081.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-11-25T09:14:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-11-25T09:14:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-11-25T09:14:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-11-25T09:14:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-11-25T09:14:01.068 INFO:teuthology.orchestra.run.smithi081.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-11-25T09:14:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-11-25T09:14:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-11-25T09:14:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-11-25T09:14:01.069 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:01.070 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:14:01.072 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python2-pyasn1-modules-0.1.9-7.el7.noarch 11/11 2019-11-25T09:14:01.072 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:01.072 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:14:01.073 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-rook.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:01.073 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:01.073 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:14:01.073 INFO:teuthology.orchestra.run.smithi153.stdout: python-cachetools.noarch 0:1.0.3-1.el7 2019-11-25T09:14:01.074 INFO:teuthology.orchestra.run.smithi153.stdout: python-jwcrypto.noarch 0:0.4.2-1.el7 2019-11-25T09:14:01.074 INFO:teuthology.orchestra.run.smithi153.stdout: python-websocket-client.noarch 0:0.56.0-3.git3c25814.el7 2019-11-25T09:14:01.074 INFO:teuthology.orchestra.run.smithi153.stdout: python2-certifi.noarch 0:2018.10.15-5.el7 2019-11-25T09:14:01.074 INFO:teuthology.orchestra.run.smithi153.stdout: python2-google-auth.noarch 1:1.1.1-5.el7 2019-11-25T09:14:01.075 INFO:teuthology.orchestra.run.smithi153.stdout: python2-kubernetes.noarch 0:10.0.1-1.el7 2019-11-25T09:14:01.075 INFO:teuthology.orchestra.run.smithi153.stdout: python2-oauthlib.noarch 0:2.0.1-8.el7 2019-11-25T09:14:01.075 INFO:teuthology.orchestra.run.smithi153.stdout: python2-pyasn1-modules.noarch 0:0.1.9-7.el7 2019-11-25T09:14:01.075 INFO:teuthology.orchestra.run.smithi153.stdout: python2-requests-oauthlib.noarch 0:0.8.0-5.el7 2019-11-25T09:14:01.076 INFO:teuthology.orchestra.run.smithi153.stdout: python2-rsa.noarch 0:3.4.1-1.el7 2019-11-25T09:14:01.076 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:01.079 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:14:01.192 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-mgr-ssh 2019-11-25T09:14:01.211 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-mgr-ssh 2019-11-25T09:14:01.428 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:01.463 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:01.463 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:14:01.468 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:14:01.468 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:01.468 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:01.488 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:01.535 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:01.536 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:14:01.537 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:14:01.537 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:01.537 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:02.105 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:02.189 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:02.432 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:14:02.432 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:02.432 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:02.439 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:14:02.464 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:02.465 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-11-25T09:14:02.465 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-11-25T09:14:02.466 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:02.466 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-11-25T09:14:02.533 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:14:02.533 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:02.533 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-mgr-ssh.noarch 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:02.536 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-remoto for package: 2:ceph-mgr-ssh-15.0.0-7614.g12b9659.el7.noarch 2019-11-25T09:14:02.566 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:02.566 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-remoto.noarch 0:0.0.25-1.el7 will be installed 2019-11-25T09:14:02.567 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python-execnet >= 1.2.0 for package: python-remoto-0.0.25-1.el7.noarch 2019-11-25T09:14:02.567 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:02.568 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-execnet.noarch 0:1.2.0-7.el7 will be installed 2019-11-25T09:14:02.831 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:02.933 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:03.054 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:03.055 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:14:03.055 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:03.056 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:03.056 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2019-11-25T09:14:03.056 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:03.056 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:14:03.056 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-ssh noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 30 k 2019-11-25T09:14:03.056 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:14:03.056 INFO:teuthology.orchestra.run.smithi081.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-11-25T09:14:03.057 INFO:teuthology.orchestra.run.smithi081.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-11-25T09:14:03.057 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:03.057 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:14:03.057 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:03.057 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+2 Dependent packages) 2019-11-25T09:14:03.057 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:03.058 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 313 k 2019-11-25T09:14:03.058 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 1.1 M 2019-11-25T09:14:03.058 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:14:03.161 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:03.161 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:14:03.162 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:03.162 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:03.162 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T09:14:03.162 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:03.162 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:14:03.163 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-ssh noarch 2:15.0.0-7614.g12b9659.el7 ceph-noarch 30 k 2019-11-25T09:14:03.163 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:14:03.163 INFO:teuthology.orchestra.run.smithi153.stdout: python-execnet noarch 1.2.0-7.el7 epel 261 k 2019-11-25T09:14:03.163 INFO:teuthology.orchestra.run.smithi153.stdout: python-remoto noarch 0.0.25-1.el7 epel 22 k 2019-11-25T09:14:03.163 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:03.163 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:14:03.163 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:03.164 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+2 Dependent packages) 2019-11-25T09:14:03.164 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:03.164 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 313 k 2019-11-25T09:14:03.165 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 1.1 M 2019-11-25T09:14:03.165 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:14:03.586 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:14:03.586 INFO:teuthology.orchestra.run.smithi081.stdout:Total 591 kB/s | 313 kB 00:00 2019-11-25T09:14:03.590 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:14:03.599 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:14:03.681 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:14:03.681 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:14:03.708 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:14:03.708 INFO:teuthology.orchestra.run.smithi153.stdout:Total 583 kB/s | 313 kB 00:00 2019-11-25T09:14:03.708 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:14:03.713 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:14:03.789 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:14:03.789 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:14:04.494 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-11-25T09:14:04.607 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-execnet-1.2.0-7.el7.noarch 1/3 2019-11-25T09:14:04.747 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-11-25T09:14:04.861 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python-remoto-0.0.25-1.el7.noarch 2/3 2019-11-25T09:14:05.100 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7614.g12b9659.el7.noarch 3/3 2019-11-25T09:14:05.156 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-mgr-ssh-15.0.0-7614.g12b9659.el7.noarch 3/3 2019-11-25T09:14:05.207 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-11-25T09:14:05.243 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-remoto-0.0.25-1.el7.noarch 1/3 2019-11-25T09:14:05.338 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-11-25T09:14:05.377 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python-execnet-1.2.0-7.el7.noarch 2/3 2019-11-25T09:14:05.846 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7614.g12b9659.el7.noarch 3/3 2019-11-25T09:14:05.847 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:05.847 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:14:05.847 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:05.847 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:05.847 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:14:05.847 INFO:teuthology.orchestra.run.smithi081.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-11-25T09:14:05.848 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:05.848 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:14:05.964 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install ceph-fuse 2019-11-25T09:14:06.114 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-mgr-ssh-15.0.0-7614.g12b9659.el7.noarch 3/3 2019-11-25T09:14:06.114 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:06.115 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:14:06.115 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-mgr-ssh.noarch 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:06.115 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:06.115 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:14:06.115 INFO:teuthology.orchestra.run.smithi153.stdout: python-execnet.noarch 0:1.2.0-7.el7 python-remoto.noarch 0:0.0.25-1.el7 2019-11-25T09:14:06.115 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:06.116 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:14:06.141 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:06.182 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:06.182 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:14:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:14:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:06.183 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:06.266 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install ceph-fuse 2019-11-25T09:14:06.523 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:06.563 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:06.576 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:14:06.577 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:14:06.577 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:06.577 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:06.780 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:07.100 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:14:07.101 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:07.101 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:07.111 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python36 for package: 2:ceph-fuse-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:14:07.143 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:14:07.144 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:07.144 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-11-25T09:14:07.145 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-11-25T09:14:07.147 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-11-25T09:14:07.147 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-11-25T09:14:07.148 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-11-25T09:14:07.149 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:07.149 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-11-25T09:14:07.155 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-11-25T09:14:07.155 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-11-25T09:14:07.217 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:07.516 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:07.552 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:14:07.552 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:07.552 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package ceph-fuse.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:07.562 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python36 for package: 2:ceph-fuse-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:14:07.594 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: /usr/bin/python3 for package: 2:ceph-fuse-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:14:07.595 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:07.595 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python3.x86_64 0:3.6.8-10.el7 will be installed 2019-11-25T09:14:07.596 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python3-libs(x86-64) = 3.6.8-10.el7 for package: python3-3.6.8-10.el7.x86_64 2019-11-25T09:14:07.597 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python3-setuptools for package: python3-3.6.8-10.el7.x86_64 2019-11-25T09:14:07.598 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: python3-pip for package: python3-3.6.8-10.el7.x86_64 2019-11-25T09:14:07.599 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libpython3.6m.so.1.0()(64bit) for package: python3-3.6.8-10.el7.x86_64 2019-11-25T09:14:07.600 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:07.600 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python3-libs.x86_64 0:3.6.8-10.el7 will be installed 2019-11-25T09:14:07.606 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python3-pip.noarch 0:9.0.3-5.el7 will be installed 2019-11-25T09:14:07.606 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python3-setuptools.noarch 0:39.2.0-10.el7 will be installed 2019-11-25T09:14:07.738 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:07.739 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:14:07.740 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:07.740 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:07.740 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository 2019-11-25T09:14:07.740 INFO:teuthology.orchestra.run.smithi081.stdout: Size 2019-11-25T09:14:07.740 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:07.741 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:14:07.741 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse x86_64 2:15.0.0-7614.g12b9659.el7 ceph 566 k 2019-11-25T09:14:07.741 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:14:07.741 INFO:teuthology.orchestra.run.smithi081.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-11-25T09:14:07.742 INFO:teuthology.orchestra.run.smithi081.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-11-25T09:14:07.742 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-11-25T09:14:07.742 INFO:teuthology.orchestra.run.smithi081.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-11-25T09:14:07.742 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:07.742 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:14:07.742 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:07.743 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+4 Dependent packages) 2019-11-25T09:14:07.743 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:07.743 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 10 M 2019-11-25T09:14:07.743 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 49 M 2019-11-25T09:14:07.743 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:14:07.969 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:08.195 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:08.195 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:14:08.196 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:08.196 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:08.196 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository 2019-11-25T09:14:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: Size 2019-11-25T09:14:08.197 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:08.197 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:14:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse x86_64 2:15.0.0-7614.g12b9659.el7 ceph 566 k 2019-11-25T09:14:08.197 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:14:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: python3 x86_64 3.6.8-10.el7 base 69 k 2019-11-25T09:14:08.197 INFO:teuthology.orchestra.run.smithi153.stdout: python3-libs x86_64 3.6.8-10.el7 base 7.0 M 2019-11-25T09:14:08.198 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pip noarch 9.0.3-5.el7 base 1.8 M 2019-11-25T09:14:08.198 INFO:teuthology.orchestra.run.smithi153.stdout: python3-setuptools noarch 39.2.0-10.el7 base 629 k 2019-11-25T09:14:08.198 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:08.198 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:14:08.199 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:08.199 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+4 Dependent packages) 2019-11-25T09:14:08.199 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:08.199 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 10 M 2019-11-25T09:14:08.199 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 49 M 2019-11-25T09:14:08.199 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:14:08.299 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:14:08.299 INFO:teuthology.orchestra.run.smithi081.stdout:Total 18 MB/s | 10 MB 00:00 2019-11-25T09:14:08.310 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:14:08.327 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:14:08.460 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:14:08.461 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:14:08.802 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:14:08.803 INFO:teuthology.orchestra.run.smithi153.stdout:Total 16 MB/s | 10 MB 00:00 2019-11-25T09:14:08.828 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:14:08.848 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:14:08.971 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:14:08.972 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:14:10.682 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-11-25T09:14:11.173 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-11-25T09:14:11.345 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-libs-3.6.8-10.el7.x86_64 1/5 2019-11-25T09:14:11.849 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-3.6.8-10.el7.x86_64 2/5 2019-11-25T09:14:12.018 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-11-25T09:14:12.508 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-11-25T09:14:12.704 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-setuptools-39.2.0-10.el7.noarch 3/5 2019-11-25T09:14:12.751 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:ceph-fuse-15.0.0-7614.g12b9659.el7.x86_64 5/5 2019-11-25T09:14:12.892 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 1/5 2019-11-25T09:14:12.993 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:ceph-fuse-15.0.0-7614.g12b9659.el7.x86_64 2/5 2019-11-25T09:14:13.085 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-11-25T09:14:13.180 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : python3-pip-9.0.3-5.el7.noarch 4/5 2019-11-25T09:14:13.194 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-11-25T09:14:13.489 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:ceph-fuse-15.0.0-7614.g12b9659.el7.x86_64 5/5 2019-11-25T09:14:13.623 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-setuptools-39.2.0-10.el7.noarch 1/5 2019-11-25T09:14:13.710 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:ceph-fuse-15.0.0-7614.g12b9659.el7.x86_64 2/5 2019-11-25T09:14:13.711 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-11-25T09:14:13.711 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:13.711 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:14:13.711 INFO:teuthology.orchestra.run.smithi081.stdout: ceph-fuse.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:13.711 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:13.711 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:14:13.711 INFO:teuthology.orchestra.run.smithi081.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-11-25T09:14:13.712 INFO:teuthology.orchestra.run.smithi081.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-11-25T09:14:13.712 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:13.712 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:14:13.789 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-libs-3.6.8-10.el7.x86_64 3/5 2019-11-25T09:14:13.825 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install libcephfs2 2019-11-25T09:14:13.873 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-3.6.8-10.el7.x86_64 4/5 2019-11-25T09:14:14.085 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:14.130 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:14:14.132 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:14:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:14.133 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:14.353 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : python3-pip-9.0.3-5.el7.noarch 5/5 2019-11-25T09:14:14.353 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:14.353 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:14:14.353 INFO:teuthology.orchestra.run.smithi153.stdout: ceph-fuse.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:14.353 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:14.354 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:14:14.354 INFO:teuthology.orchestra.run.smithi153.stdout: python3.x86_64 0:3.6.8-10.el7 python3-libs.x86_64 0:3.6.8-10.el7 2019-11-25T09:14:14.354 INFO:teuthology.orchestra.run.smithi153.stdout: python3-pip.noarch 0:9.0.3-5.el7 python3-setuptools.noarch 0:39.2.0-10.el7 2019-11-25T09:14:14.354 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:14.354 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:14:14.473 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install libcephfs2 2019-11-25T09:14:14.707 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:14.747 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:14.749 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:14:14.749 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:14:14.749 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:14.750 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:14.778 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:15.115 INFO:teuthology.orchestra.run.smithi081.stdout:Package 2:libcephfs2-15.0.0-7614.g12b9659.el7.x86_64 already installed and latest version 2019-11-25T09:14:15.115 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do 2019-11-25T09:14:15.174 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install libcephfs-devel 2019-11-25T09:14:15.378 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:15.406 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:15.415 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:15.417 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:14:15.417 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:14:15.418 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:15.418 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:15.735 INFO:teuthology.orchestra.run.smithi153.stdout:Package 2:libcephfs2-15.0.0-7614.g12b9659.el7.x86_64 already installed and latest version 2019-11-25T09:14:15.735 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do 2019-11-25T09:14:15.792 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install libcephfs-devel 2019-11-25T09:14:15.981 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:15.992 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:16.016 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:16.017 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:14:16.017 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:14:16.017 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:16.018 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:16.313 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:14:16.314 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:16.314 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:16.317 INFO:teuthology.orchestra.run.smithi081.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7614.g12b9659.el7 for package: 2:libcephfs-devel-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:14:16.344 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:16.344 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package librados-devel.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:16.630 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:16.717 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:16.966 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:14:16.966 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:16.966 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libcephfs-devel.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:16.967 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: librados-devel = 2:15.0.0-7614.g12b9659.el7 for package: 2:libcephfs-devel-15.0.0-7614.g12b9659.el7.x86_64 2019-11-25T09:14:16.967 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:16.967 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:14:16.967 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:16.967 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:16.967 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository 2019-11-25T09:14:16.968 INFO:teuthology.orchestra.run.smithi081.stdout: Size 2019-11-25T09:14:16.968 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:16.968 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:14:16.968 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel x86_64 2:15.0.0-7614.g12b9659.el7 ceph 17 k 2019-11-25T09:14:16.968 INFO:teuthology.orchestra.run.smithi081.stdout:Installing for dependencies: 2019-11-25T09:14:16.981 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel x86_64 2:15.0.0-7614.g12b9659.el7 ceph 93 k 2019-11-25T09:14:16.981 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:16.981 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:14:16.981 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:16.981 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package (+1 Dependent package) 2019-11-25T09:14:16.982 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:16.982 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 109 k 2019-11-25T09:14:16.982 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 398 k 2019-11-25T09:14:16.982 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:14:16.986 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:16.986 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package librados-devel.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:17.290 INFO:teuthology.orchestra.run.smithi081.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:14:17.290 INFO:teuthology.orchestra.run.smithi081.stdout:Total 317 kB/s | 109 kB 00:00 2019-11-25T09:14:17.293 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:14:17.315 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:14:17.317 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:14:17.318 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:14:17.361 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:17.588 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:17.589 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:14:17.589 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:17.590 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository 2019-11-25T09:14:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: Size 2019-11-25T09:14:17.590 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:17.590 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:14:17.590 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel x86_64 2:15.0.0-7614.g12b9659.el7 ceph 17 k 2019-11-25T09:14:17.591 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:14:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel x86_64 2:15.0.0-7614.g12b9659.el7 ceph 93 k 2019-11-25T09:14:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:17.591 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:14:17.591 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:17.591 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package (+1 Dependent package) 2019-11-25T09:14:17.591 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:17.592 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 109 k 2019-11-25T09:14:17.592 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 398 k 2019-11-25T09:14:17.592 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:14:17.890 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:librados-devel-15.0.0-7614.g12b9659.el7.x86_64 1/2 2019-11-25T09:14:17.940 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:14:17.940 INFO:teuthology.orchestra.run.smithi153.stdout:Total 313 kB/s | 109 kB 00:00 2019-11-25T09:14:17.942 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:14:17.962 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:14:17.964 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:14:17.965 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:14:18.168 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:libcephfs-devel-15.0.0-7614.g12b9659.el7.x86_64 2/2 2019-11-25T09:14:18.257 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:librados-devel-15.0.0-7614.g12b9659.el7.x86_64 1/2 2019-11-25T09:14:18.553 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:librados-devel-15.0.0-7614.g12b9659.el7.x86_64 1/2 2019-11-25T09:14:18.727 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:libcephfs-devel-15.0.0-7614.g12b9659.el7.x86_64 2/2 2019-11-25T09:14:18.727 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:18.727 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:14:18.727 INFO:teuthology.orchestra.run.smithi081.stdout: libcephfs-devel.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:18.727 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:18.727 INFO:teuthology.orchestra.run.smithi081.stdout:Dependency Installed: 2019-11-25T09:14:18.728 INFO:teuthology.orchestra.run.smithi081.stdout: librados-devel.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:18.728 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:18.728 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:14:18.815 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:libcephfs-devel-15.0.0-7614.g12b9659.el7.x86_64 2/2 2019-11-25T09:14:18.848 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install librados2 2019-11-25T09:14:18.921 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:librados-devel-15.0.0-7614.g12b9659.el7.x86_64 1/2 2019-11-25T09:14:19.125 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:19.165 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:19.166 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:14:19.166 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:14:19.167 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:19.167 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:19.422 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:libcephfs-devel-15.0.0-7614.g12b9659.el7.x86_64 2/2 2019-11-25T09:14:19.422 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:19.422 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:14:19.423 INFO:teuthology.orchestra.run.smithi153.stdout: libcephfs-devel.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:19.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:19.423 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:14:19.424 INFO:teuthology.orchestra.run.smithi153.stdout: librados-devel.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:19.424 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:19.424 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:14:19.550 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install librados2 2019-11-25T09:14:19.752 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:19.790 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:19.792 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:14:19.792 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:14:19.792 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:19.792 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:19.804 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:20.147 INFO:teuthology.orchestra.run.smithi081.stdout:Package 2:librados2-15.0.0-7614.g12b9659.el7.x86_64 already installed and latest version 2019-11-25T09:14:20.147 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do 2019-11-25T09:14:20.214 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install librbd1 2019-11-25T09:14:20.374 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:20.408 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:20.409 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:14:20.410 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:14:20.410 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:20.410 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:20.457 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:20.783 INFO:teuthology.orchestra.run.smithi153.stdout:Package 2:librados2-15.0.0-7614.g12b9659.el7.x86_64 already installed and latest version 2019-11-25T09:14:20.783 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do 2019-11-25T09:14:20.844 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install librbd1 2019-11-25T09:14:20.988 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:21.093 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:21.139 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:21.140 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:14:21.140 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:14:21.141 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:21.141 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:21.308 INFO:teuthology.orchestra.run.smithi081.stdout:Package 2:librbd1-15.0.0-7614.g12b9659.el7.x86_64 already installed and latest version 2019-11-25T09:14:21.309 INFO:teuthology.orchestra.run.smithi081.stdout:Nothing to do 2019-11-25T09:14:21.368 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install python-ceph 2019-11-25T09:14:21.633 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:21.679 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:21.680 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:14:21.680 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:14:21.680 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:21.681 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:21.789 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:22.130 INFO:teuthology.orchestra.run.smithi153.stdout:Package 2:librbd1-15.0.0-7614.g12b9659.el7.x86_64 already installed and latest version 2019-11-25T09:14:22.131 INFO:teuthology.orchestra.run.smithi153.stdout:Nothing to do 2019-11-25T09:14:22.189 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install python-ceph 2019-11-25T09:14:22.326 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:22.397 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:22.441 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:22.442 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:14:22.442 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:14:22.443 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:22.443 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:22.667 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:14:22.668 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:22.668 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:23.037 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:23.082 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:23.258 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:23.258 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:14:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:23.259 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository 2019-11-25T09:14:23.259 INFO:teuthology.orchestra.run.smithi081.stdout: Size 2019-11-25T09:14:23.260 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:23.260 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:14:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: python-ceph-compat x86_64 2:15.0.0-7614.g12b9659.el7 ceph 2.2 k 2019-11-25T09:14:23.260 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:23.261 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:14:23.261 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:23.261 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2019-11-25T09:14:23.261 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:23.261 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 2.2 k 2019-11-25T09:14:23.261 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 0 2019-11-25T09:14:23.261 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:14:23.416 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:14:23.416 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:23.416 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package python-ceph-compat.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:23.543 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:14:23.543 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:14:23.549 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:14:23.549 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:14:23.785 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:24.015 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:24.015 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:14:24.016 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:24.016 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:24.016 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository 2019-11-25T09:14:24.017 INFO:teuthology.orchestra.run.smithi153.stdout: Size 2019-11-25T09:14:24.017 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:24.017 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:14:24.017 INFO:teuthology.orchestra.run.smithi153.stdout: python-ceph-compat x86_64 2:15.0.0-7614.g12b9659.el7 ceph 2.2 k 2019-11-25T09:14:24.017 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:24.017 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:14:24.018 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:24.018 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2019-11-25T09:14:24.018 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:24.018 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 2.2 k 2019-11-25T09:14:24.018 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 0 2019-11-25T09:14:24.018 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:14:24.082 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:python-ceph-compat-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:14:24.311 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:14:24.312 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:14:24.313 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:14:24.314 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:14:24.547 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:python-ceph-compat-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:14:24.547 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:24.548 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:14:24.548 INFO:teuthology.orchestra.run.smithi081.stdout: python-ceph-compat.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:24.548 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:24.548 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:14:24.657 INFO:teuthology.orchestra.run.smithi081:> sudo yum -y install rbd-fuse 2019-11-25T09:14:24.861 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:python-ceph-compat-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:14:24.913 INFO:teuthology.orchestra.run.smithi081.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:24.957 INFO:teuthology.orchestra.run.smithi081.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:24.958 INFO:teuthology.orchestra.run.smithi081.stdout: * base: packages.oit.ncsu.edu 2019-11-25T09:14:24.958 INFO:teuthology.orchestra.run.smithi081.stdout: * epel: download-cc-rdu01.fedoraproject.org 2019-11-25T09:14:24.958 INFO:teuthology.orchestra.run.smithi081.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:24.959 INFO:teuthology.orchestra.run.smithi081.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:25.411 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:python-ceph-compat-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:14:25.412 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:25.412 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:14:25.412 INFO:teuthology.orchestra.run.smithi153.stdout: python-ceph-compat.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:25.412 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:25.412 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:14:25.541 INFO:teuthology.orchestra.run.smithi153:> sudo yum -y install rbd-fuse 2019-11-25T09:14:25.600 INFO:teuthology.orchestra.run.smithi081.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:25.769 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:14:25.813 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:14:25.814 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:14:25.814 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:14:25.814 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:14:25.815 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:14:25.936 INFO:teuthology.orchestra.run.smithi081.stdout:Resolving Dependencies 2019-11-25T09:14:25.936 INFO:teuthology.orchestra.run.smithi081.stdout:--> Running transaction check 2019-11-25T09:14:25.936 INFO:teuthology.orchestra.run.smithi081.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:26.309 INFO:teuthology.orchestra.run.smithi081.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:26.475 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:14:26.534 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:26.534 INFO:teuthology.orchestra.run.smithi081.stdout:Dependencies Resolved 2019-11-25T09:14:26.535 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:26.535 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:26.535 INFO:teuthology.orchestra.run.smithi081.stdout: Package Arch Version Repository Size 2019-11-25T09:14:26.535 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:26.536 INFO:teuthology.orchestra.run.smithi081.stdout:Installing: 2019-11-25T09:14:26.536 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse x86_64 2:15.0.0-7614.g12b9659.el7 ceph 76 k 2019-11-25T09:14:26.536 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:26.536 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction Summary 2019-11-25T09:14:26.536 INFO:teuthology.orchestra.run.smithi081.stdout:================================================================================ 2019-11-25T09:14:26.536 INFO:teuthology.orchestra.run.smithi081.stdout:Install 1 Package 2019-11-25T09:14:26.536 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:26.537 INFO:teuthology.orchestra.run.smithi081.stdout:Total download size: 76 k 2019-11-25T09:14:26.537 INFO:teuthology.orchestra.run.smithi081.stdout:Installed size: 200 k 2019-11-25T09:14:26.537 INFO:teuthology.orchestra.run.smithi081.stdout:Downloading packages: 2019-11-25T09:14:26.814 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:14:26.815 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:14:26.815 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package rbd-fuse.x86_64 2:15.0.0-7614.g12b9659.el7 will be installed 2019-11-25T09:14:26.852 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction check 2019-11-25T09:14:26.872 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction test 2019-11-25T09:14:26.874 INFO:teuthology.orchestra.run.smithi081.stdout:Transaction test succeeded 2019-11-25T09:14:26.874 INFO:teuthology.orchestra.run.smithi081.stdout:Running transaction 2019-11-25T09:14:27.191 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:14:27.403 INFO:teuthology.orchestra.run.smithi081.stdout: Installing : 2:rbd-fuse-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:14:27.421 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:27.421 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:14:27.422 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:27.422 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:27.423 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T09:14:27.423 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:27.423 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:14:27.423 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse x86_64 2:15.0.0-7614.g12b9659.el7 ceph 76 k 2019-11-25T09:14:27.423 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:27.424 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:14:27.424 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:14:27.424 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2019-11-25T09:14:27.424 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:27.425 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 76 k 2019-11-25T09:14:27.425 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 200 k 2019-11-25T09:14:27.425 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:14:27.760 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:14:27.780 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:14:27.782 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:14:27.783 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:14:27.876 INFO:teuthology.orchestra.run.smithi081.stdout: Verifying : 2:rbd-fuse-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:14:27.876 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:27.877 INFO:teuthology.orchestra.run.smithi081.stdout:Installed: 2019-11-25T09:14:27.877 INFO:teuthology.orchestra.run.smithi081.stdout: rbd-fuse.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:27.877 INFO:teuthology.orchestra.run.smithi081.stdout: 2019-11-25T09:14:27.877 INFO:teuthology.orchestra.run.smithi081.stdout:Complete! 2019-11-25T09:14:27.993 DEBUG:teuthology.parallel:result is None 2019-11-25T09:14:28.265 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : 2:rbd-fuse-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:14:28.772 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : 2:rbd-fuse-15.0.0-7614.g12b9659.el7.x86_64 1/1 2019-11-25T09:14:28.773 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:28.773 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:14:28.773 INFO:teuthology.orchestra.run.smithi153.stdout: rbd-fuse.x86_64 2:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:28.773 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:14:28.773 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:14:28.883 DEBUG:teuthology.parallel:result is None 2019-11-25T09:14:28.883 INFO:teuthology.orchestra.run.smithi153:> sudo lsb_release -is 2019-11-25T09:14:29.031 INFO:teuthology.orchestra.run.smithi153.stdout:CentOS 2019-11-25T09:14:29.033 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-25T09:14:29.033 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=12b9659be24ae0e3d791f80d81504ab2770c1343 2019-11-25T09:14:29.193 INFO:teuthology.orchestra.run.smithi153:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-11-25T09:14:29.328 INFO:teuthology.orchestra.run.smithi153.stdout:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:29.328 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7614.g12b9659.el7 2019-11-25T09:14:29.329 INFO:teuthology.task.install:The correct ceph version 15.0.0-7614.g12b9659 is installed. 2019-11-25T09:14:29.329 INFO:teuthology.orchestra.run.smithi081:> sudo lsb_release -is 2019-11-25T09:14:29.483 INFO:teuthology.orchestra.run.smithi081.stdout:CentOS 2019-11-25T09:14:29.486 DEBUG:teuthology.misc:System to be installed: CentOS 2019-11-25T09:14:29.486 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=notcmalloc&distros=centos%2F7%2Fx86_64&sha1=12b9659be24ae0e3d791f80d81504ab2770c1343 2019-11-25T09:14:29.641 INFO:teuthology.orchestra.run.smithi081:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2019-11-25T09:14:29.780 INFO:teuthology.orchestra.run.smithi081.stdout:15.0.0-7614.g12b9659.el7 2019-11-25T09:14:29.780 INFO:teuthology.packaging:The installed version of ceph is 15.0.0-7614.g12b9659.el7 2019-11-25T09:14:29.780 INFO:teuthology.task.install:The correct ceph version 15.0.0-7614.g12b9659 is installed. 2019-11-25T09:14:29.780 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2019-11-25T09:14:29.781 INFO:teuthology.orchestra.run.smithi153:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-11-25T09:14:29.902 INFO:teuthology.orchestra.run.smithi081:> sudo sh -c 'cat > /home/ubuntu/cephtest/valgrind.supp' 2019-11-25T09:14:30.025 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2019-11-25T09:14:30.025 INFO:teuthology.orchestra.run.smithi153:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-11-25T09:14:30.149 INFO:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-11-25T09:14:30.242 INFO:teuthology.orchestra.run.smithi081:> sudo sh -c 'cat > /usr/bin/daemon-helper' 2019-11-25T09:14:30.361 INFO:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/daemon-helper 2019-11-25T09:14:30.464 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2019-11-25T09:14:30.464 INFO:teuthology.orchestra.run.smithi153:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-11-25T09:14:30.588 INFO:teuthology.orchestra.run.smithi153:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-11-25T09:14:30.683 INFO:teuthology.orchestra.run.smithi081:> sudo sh -c 'cat > /usr/bin/adjust-ulimits' 2019-11-25T09:14:30.806 INFO:teuthology.orchestra.run.smithi081:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2019-11-25T09:14:30.906 INFO:teuthology.run_tasks:Running task ceph... 2019-11-25T09:14:31.032 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2019-11-25T09:14:31.032 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 777 /var/log/ceph 2019-11-25T09:14:31.038 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 777 /var/log/ceph 2019-11-25T09:14:31.153 INFO:tasks.ceph:Disabling ceph logrotate... 2019-11-25T09:14:31.153 INFO:teuthology.orchestra.run.smithi081:> sudo rm -f -- /etc/logrotate.d/ceph 2019-11-25T09:14:31.158 INFO:teuthology.orchestra.run.smithi153:> sudo rm -f -- /etc/logrotate.d/ceph 2019-11-25T09:14:31.265 INFO:tasks.ceph:Creating extra log directories... 2019-11-25T09:14:31.265 INFO:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-11-25T09:14:31.270 INFO:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2019-11-25T09:14:31.373 INFO:tasks.ceph:Creating ceph cluster ceph... 2019-11-25T09:14:31.373 INFO:tasks.ceph:config {'mon_bind_addrvec': True, 'block_journal': None, 'fs': 'xfs', 'log_whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'skip_mgr_daemons': False, 'cpu_profile': set([]), 'mkfs_options': None, 'tmpfs_journal': None, 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mount_options': None, 'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'debug ms': 1, 'debug journal': 20, 'osd shutdown pgref assert': True, 'debug osd': 25, 'debug filestore': 20, 'osd objectstore': 'filestore', 'osd sloppy crc': True}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}} 2019-11-25T09:14:31.374 INFO:tasks.ceph:ctx.config {'kernel': {'sha1': 'distro', 'kdb': True}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.lock_machines': (2, 'smithi')}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'sha1': 'distro', 'kdb': True}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'flavor': 'notcmalloc', 'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343'}}, {'ceph': None}, {'openssl_keys': None}, {'rgw': {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh']}}}, {'ragweed': {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}}}, {'s3tests-java': {'client.0': {'force-repo': 'https://github.com/ceph/java_s3tests.git', 'force-branch': 'master'}}}, {'s3tests': {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master'}}}, {'swift': {'client.0': {'rgw_server': 'client.0'}}}], 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_wip-yuval-add-kafka-security2/qa', 'verbose': False, 'worker_log': '/home/teuthworker/archive/worker_logs/worker.smithi.9224', 'owner': 'scheduled_yuvalif@teuthology', 'openstack': [{'volumes': {'count': 4, 'size': 10}}], 'targets': {'smithi153.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDbJ3CA33GkUFO5ip99XQlSODRE6lM/dGjTcdtOM3LFHo8ORrhEGq6shai2YwAzGWaPyBaIKHEBHrpou7Rnkd3wHMtJRL2I0V4MOV/xOKCqWhSkowB2zlt53QSjR+2Pn0o558mhE/aWKcqAsd9OLU8mULVMutLYY3krVOK4ompYHGkJc+KF3x8KvYJ1H4PCp2oWYYbXWmqAb7+636SaD/fgmDSICeUhsHcWYpIsLdGDAiV47zyCvcO/Kyn00KdUpDDZkII36EKEO4fLCZ9yeQ3f8Hcn1DmJtnFTMIJAWm4xDMtjz+T/OEkXYyJkWTqXdxEATcgmps2VcwWfvgGxZ4AP', 'smithi081.front.sepia.ceph.com': 'ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQDulLtKBKKywL9l8dfPo+H+fQDMdFzYGQ7+KrMuzE1hl2i63HEGzGR2FHjQHLDiwJ4CQvJTKBiOfDaRzQaLWaPpT5S6hzxvUvZE6EvOSwFS5Ql3OWrx3XwJCn91GW6I8PxO7Tjj9CORQFjIym5HyKwrsP97/1ZohFL+BRHZMuSxTazM07PHdTf+NtZPqtaeWoydHb8z6R143o0VcpLe9Nycbv5fWyE5GgSBomd9G4n+uHDCJ9owpWZKx8WmvtSb7D4cZW+I5mSPCmzTIwh7cvXYl9rF72PAcentXWHRWJ5nBWe5IzI3yNg5yzg8yIKfH8BCoO2pykVeDErsBeb/qEkf'}, 'job_id': '4542041', 'tube': 'smithi', 'suite_branch': 'wip-yuval-add-kafka-security2', 'priority': 100, 'first_in_suite': False, 'branch': 'wip-yuval-add-kafka-security2', 'suite': 'rgw:verify', 'email': 'yuvalif@yahoo.com', 'archive_path': '/home/teuthworker/archive/yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi/4542041', 'description': 'rgw:verify/{clusters/fixed-2.yaml frontend/civetweb.yaml msgr-failures/few.yaml objectstore/filestore-xfs.yaml overrides.yaml proto/http.yaml rgw_pool_type/ec.yaml tasks/{0-install.yaml cls.yaml ragweed.yaml s3tests-java.yaml s3tests.yaml swift.yaml} validater/valgrind.yaml}', 'nuke-on-error': True, 'repo': 'https://github.com/ceph/ceph-ci.git', 'last_in_suite': False, 'machine_type': 'smithi', 'teuthology_branch': 'master', 'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343', 'name': 'yuvalif-2019-11-25_06:14:24-rgw:verify-wip-yuval-add-kafka-security2-distro-basic-smithi', 'roles': [['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'], ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1']], 'suite_relpath': 'qa', 'suite_repo': 'git://git.ceph.com/ceph-ci.git', 'suite_sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343', 'os_type': 'centos', 'overrides': {'ceph': {'conf': {'global': {'osd heartbeat grace': 40, 'ms inject socket failures': 5000, 'osd_max_pg_log_entries': 10, 'mon client directed command retry': 5, 'osd_min_pg_log_entries': 10}, 'osd': {'debug ms': 1, 'debug journal': 20, 'osd shutdown pgref assert': True, 'debug osd': 25, 'debug filestore': 20, 'osd objectstore': 'filestore', 'osd sloppy crc': True}, 'mon': {'debug mon': 20, 'debug paxos': 20, 'mon osd crush smoke test': False, 'debug ms': 1}, 'client': {'debug rgw': 20, 'rgw crypt s3 kms backend': 'testing', 'rgw crypt require ssl': False, 'rgw crypt s3 kms encryption keys': 'testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo=', 'rgw lc debug interval': 10}}, 'log-whitelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(OSD_SLOW_PING_TIME'], 'valgrind': {'mds': ['--tool=memcheck'], 'mon': ['--tool=memcheck', '--leak-check=full', '--show-reachable=yes'], 'osd': ['--tool=memcheck']}, 'fs': 'xfs', 'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343'}, 's3tests': {'slow_backend': True}, 'ceph-deploy': {'fs': 'xfs', 'filestore': True, 'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {'osd default pool size': 2}, 'osd': {'osd sloppy crc': True, 'osd objectstore': 'filestore'}}}, 'install': {'ceph': {'flavor': 'notcmalloc', 'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343'}}, 'rgw': {'frontend': 'civetweb', 'compression type': 'random', 'storage classes': 'LUKEWARM, FROZEN', 'ec-data-pool': True}, 'admin_socket': {'branch': 'wip-yuval-add-kafka-security2'}, 'workunit': {'sha1': '12b9659be24ae0e3d791f80d81504ab2770c1343', 'branch': 'wip-yuval-add-kafka-security2'}}} 2019-11-25T09:14:31.374 INFO:teuthology.orchestra.run.smithi081:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-11-25T09:14:31.380 INFO:teuthology.orchestra.run.smithi153:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2019-11-25T09:14:31.459 INFO:teuthology.orchestra.run.smithi081:> sudo install -d -m0777 -- /var/run/ceph 2019-11-25T09:14:31.464 INFO:teuthology.orchestra.run.smithi153:> sudo install -d -m0777 -- /var/run/ceph 2019-11-25T09:14:31.652 DEBUG:teuthology.orchestra.remote:smithi153:/scratch_devs is 72B 2019-11-25T09:14:31.740 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-11-25T09:14:31.741 INFO:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-11-25T09:14:31.827 INFO:teuthology.orchestra.run.smithi153.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-11-25T09:14:31.827 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-25T09:14:31.827 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 98981 Links: 1 2019-11-25T09:14:31.828 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-25T09:14:31.828 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2019-11-25T09:14:31.828 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2019-11-25 09:13:22.060151409 +0000 2019-11-25T09:14:31.828 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2019-11-25 09:11:37.653814629 +0000 2019-11-25T09:14:31.828 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2019-11-25 09:11:37.653814629 +0000 2019-11-25T09:14:31.828 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2019-11-25T09:14:31.857 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2019-11-25T09:14:31.857 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2019-11-25T09:14:31.858 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes (512 B) copied, 0.000379552 s, 1.3 MB/s 2019-11-25T09:14:31.858 INFO:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-11-25T09:14:31.931 INFO:teuthology.orchestra.run.smithi153.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-11-25T09:14:31.931 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-25T09:14:31.931 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 93057 Links: 1 2019-11-25T09:14:31.931 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-25T09:14:31.931 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2019-11-25T09:14:31.932 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2019-11-25 09:13:22.060151409 +0000 2019-11-25T09:14:31.932 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2019-11-25 09:11:38.146821714 +0000 2019-11-25T09:14:31.932 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2019-11-25 09:11:38.146821714 +0000 2019-11-25T09:14:31.932 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2019-11-25T09:14:31.952 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2019-11-25T09:14:31.953 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2019-11-25T09:14:31.953 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes (512 B) copied, 0.000341939 s, 1.5 MB/s 2019-11-25T09:14:31.956 INFO:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-11-25T09:14:32.020 INFO:teuthology.orchestra.run.smithi153.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-11-25T09:14:32.020 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-25T09:14:32.020 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 99029 Links: 1 2019-11-25T09:14:32.020 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-25T09:14:32.021 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2019-11-25T09:14:32.021 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2019-11-25 09:13:22.060151409 +0000 2019-11-25T09:14:32.021 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2019-11-25 09:11:38.606828324 +0000 2019-11-25T09:14:32.021 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2019-11-25 09:11:38.606828324 +0000 2019-11-25T09:14:32.021 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2019-11-25T09:14:32.039 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2019-11-25T09:14:32.039 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2019-11-25T09:14:32.039 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes (512 B) copied, 0.000341056 s, 1.5 MB/s 2019-11-25T09:14:32.042 INFO:teuthology.orchestra.run.smithi153:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-11-25T09:14:32.094 INFO:teuthology.orchestra.run.smithi153.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-11-25T09:14:32.094 INFO:teuthology.orchestra.run.smithi153.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-25T09:14:32.094 INFO:teuthology.orchestra.run.smithi153.stdout:Device: 5h/5d Inode: 109753 Links: 1 2019-11-25T09:14:32.094 INFO:teuthology.orchestra.run.smithi153.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-25T09:14:32.095 INFO:teuthology.orchestra.run.smithi153.stdout:Context: system_u:object_r:device_t:s0 2019-11-25T09:14:32.095 INFO:teuthology.orchestra.run.smithi153.stdout:Access: 2019-11-25 09:13:22.061151409 +0000 2019-11-25T09:14:32.095 INFO:teuthology.orchestra.run.smithi153.stdout:Modify: 2019-11-25 09:11:39.130835855 +0000 2019-11-25T09:14:32.095 INFO:teuthology.orchestra.run.smithi153.stdout:Change: 2019-11-25 09:11:39.130835855 +0000 2019-11-25T09:14:32.095 INFO:teuthology.orchestra.run.smithi153.stdout: Birth: - 2019-11-25T09:14:32.110 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records in 2019-11-25T09:14:32.111 INFO:teuthology.orchestra.run.smithi153.stderr:1+0 records out 2019-11-25T09:14:32.111 INFO:teuthology.orchestra.run.smithi153.stderr:512 bytes (512 B) copied, 0.000324035 s, 1.6 MB/s 2019-11-25T09:14:32.114 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-11-25T09:14:32.114 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-11-25T09:14:32.115 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-11-25T09:14:32.115 INFO:tasks.ceph:dev map: {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:32.208 DEBUG:teuthology.orchestra.remote:smithi081:/scratch_devs is 72B 2019-11-25T09:14:32.301 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-11-25T09:14:32.301 INFO:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_4 && sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_4 2019-11-25T09:14:32.380 INFO:teuthology.orchestra.run.smithi081.stdout: File: ‘/dev/vg_nvme/lv_4’ -> ‘../dm-1’ 2019-11-25T09:14:32.381 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-25T09:14:32.381 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 105729 Links: 1 2019-11-25T09:14:32.381 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-25T09:14:32.381 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2019-11-25T09:14:32.381 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2019-11-25 09:13:23.568985404 +0000 2019-11-25T09:14:32.381 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2019-11-25 09:11:36.870230018 +0000 2019-11-25T09:14:32.382 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2019-11-25 09:11:36.870230018 +0000 2019-11-25T09:14:32.382 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: - 2019-11-25T09:14:32.401 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2019-11-25T09:14:32.401 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2019-11-25T09:14:32.402 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes (512 B) copied, 0.00036252 s, 1.4 MB/s 2019-11-25T09:14:32.406 INFO:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_3 && sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_3 2019-11-25T09:14:32.550 INFO:teuthology.orchestra.run.smithi081.stdout: File: ‘/dev/vg_nvme/lv_3’ -> ‘../dm-2’ 2019-11-25T09:14:32.551 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-25T09:14:32.551 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 103873 Links: 1 2019-11-25T09:14:32.551 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-25T09:14:32.551 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2019-11-25T09:14:32.552 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2019-11-25 09:13:23.568985404 +0000 2019-11-25T09:14:32.552 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2019-11-25 09:11:37.332219390 +0000 2019-11-25T09:14:32.552 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2019-11-25 09:11:37.332219390 +0000 2019-11-25T09:14:32.553 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: - 2019-11-25T09:14:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2019-11-25T09:14:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2019-11-25T09:14:32.582 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes (512 B) copied, 0.000330242 s, 1.6 MB/s 2019-11-25T09:14:32.584 INFO:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_2 && sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_2 2019-11-25T09:14:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: File: ‘/dev/vg_nvme/lv_2’ -> ‘../dm-3’ 2019-11-25T09:14:32.648 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-25T09:14:32.649 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 98041 Links: 1 2019-11-25T09:14:32.649 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-25T09:14:32.649 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2019-11-25T09:14:32.650 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2019-11-25 09:13:23.569985404 +0000 2019-11-25T09:14:32.650 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2019-11-25 09:11:37.797208693 +0000 2019-11-25T09:14:32.650 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2019-11-25 09:11:37.797208693 +0000 2019-11-25T09:14:32.651 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: - 2019-11-25T09:14:32.673 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2019-11-25T09:14:32.673 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2019-11-25T09:14:32.674 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes (512 B) copied, 0.000304683 s, 1.7 MB/s 2019-11-25T09:14:32.677 INFO:teuthology.orchestra.run.smithi081:> stat /dev/vg_nvme/lv_1 && sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 && ! mount | grep -q /dev/vg_nvme/lv_1 2019-11-25T09:14:32.733 INFO:teuthology.orchestra.run.smithi081.stdout: File: ‘/dev/vg_nvme/lv_1’ -> ‘../dm-4’ 2019-11-25T09:14:32.733 INFO:teuthology.orchestra.run.smithi081.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2019-11-25T09:14:32.733 INFO:teuthology.orchestra.run.smithi081.stdout:Device: 5h/5d Inode: 98085 Links: 1 2019-11-25T09:14:32.734 INFO:teuthology.orchestra.run.smithi081.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2019-11-25T09:14:32.734 INFO:teuthology.orchestra.run.smithi081.stdout:Context: system_u:object_r:device_t:s0 2019-11-25T09:14:32.734 INFO:teuthology.orchestra.run.smithi081.stdout:Access: 2019-11-25 09:13:23.569985404 +0000 2019-11-25T09:14:32.734 INFO:teuthology.orchestra.run.smithi081.stdout:Modify: 2019-11-25 09:11:38.272197766 +0000 2019-11-25T09:14:32.734 INFO:teuthology.orchestra.run.smithi081.stdout:Change: 2019-11-25 09:11:38.272197766 +0000 2019-11-25T09:14:32.734 INFO:teuthology.orchestra.run.smithi081.stdout: Birth: - 2019-11-25T09:14:32.755 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records in 2019-11-25T09:14:32.757 INFO:teuthology.orchestra.run.smithi081.stderr:1+0 records out 2019-11-25T09:14:32.757 INFO:teuthology.orchestra.run.smithi081.stderr:512 bytes (512 B) copied, 0.000290107 s, 1.8 MB/s 2019-11-25T09:14:32.758 INFO:tasks.ceph:fs option selected, checking for scratch devs 2019-11-25T09:14:32.758 INFO:tasks.ceph:found devs: ['/dev/vg_nvme/lv_4', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_1'] 2019-11-25T09:14:32.758 WARNING:teuthology.misc:Entering get_wwn_id_map, a deprecated function that will be removed 2019-11-25T09:14:32.759 INFO:tasks.ceph:dev map: {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:32.759 INFO:tasks.ceph:Generating config... 2019-11-25T09:14:32.760 INFO:tasks.ceph:[global] osd heartbeat grace = 40 2019-11-25T09:14:32.760 INFO:tasks.ceph:[global] ms inject socket failures = 5000 2019-11-25T09:14:32.760 INFO:tasks.ceph:[global] osd_max_pg_log_entries = 10 2019-11-25T09:14:32.760 INFO:tasks.ceph:[global] mon client directed command retry = 5 2019-11-25T09:14:32.761 INFO:tasks.ceph:[global] osd_min_pg_log_entries = 10 2019-11-25T09:14:32.761 INFO:tasks.ceph:[osd] debug ms = 1 2019-11-25T09:14:32.761 INFO:tasks.ceph:[osd] debug journal = 20 2019-11-25T09:14:32.761 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2019-11-25T09:14:32.761 INFO:tasks.ceph:[osd] debug osd = 25 2019-11-25T09:14:32.762 INFO:tasks.ceph:[osd] debug filestore = 20 2019-11-25T09:14:32.762 INFO:tasks.ceph:[osd] osd objectstore = filestore 2019-11-25T09:14:32.762 INFO:tasks.ceph:[osd] osd sloppy crc = True 2019-11-25T09:14:32.762 INFO:tasks.ceph:[mon] debug mon = 20 2019-11-25T09:14:32.762 INFO:tasks.ceph:[mon] debug paxos = 20 2019-11-25T09:14:32.762 INFO:tasks.ceph:[mon] mon osd crush smoke test = False 2019-11-25T09:14:32.762 INFO:tasks.ceph:[mon] debug ms = 1 2019-11-25T09:14:32.763 INFO:tasks.ceph:[client] debug rgw = 20 2019-11-25T09:14:32.763 INFO:tasks.ceph:[client] rgw crypt s3 kms backend = testing 2019-11-25T09:14:32.763 INFO:tasks.ceph:[client] rgw crypt require ssl = False 2019-11-25T09:14:32.763 INFO:tasks.ceph:[client] rgw crypt s3 kms encryption keys = testkey-1=YmluCmJvb3N0CmJvb3N0LWJ1aWxkCmNlcGguY29uZgo= testkey-2=aWIKTWFrZWZpbGUKbWFuCm91dApzcmMKVGVzdGluZwo= 2019-11-25T09:14:32.763 INFO:tasks.ceph:[client] rgw lc debug interval = 10 2019-11-25T09:14:32.764 INFO:tasks.ceph:Setting up mon.a... 2019-11-25T09:14:32.764 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2019-11-25T09:14:32.967 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.keyring 2019-11-25T09:14:32.972 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2019-11-25T09:14:33.110 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0644 /etc/ceph/ceph.keyring 2019-11-25T09:14:33.218 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '172.21.15.153'), ('mon.c', '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]'), ('mon.b', '172.21.15.81')] 2019-11-25T09:14:33.218 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool --create --clobber --enable-all-features --add a 172.21.15.153 --addv c '[v2:172.21.15.153:3301,v1:172.21.15.153:6790]' --add b 172.21.15.81 --print /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:33.343 INFO:teuthology.orchestra.run.smithi153.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:33.343 INFO:teuthology.orchestra.run.smithi153.stdout:monmaptool: generated fsid 89933446-017a-4383-871e-aaf9415b6b73 2019-11-25T09:14:33.344 INFO:teuthology.orchestra.run.smithi153.stdout:epoch 0 2019-11-25T09:14:33.344 INFO:teuthology.orchestra.run.smithi153.stdout:fsid 89933446-017a-4383-871e-aaf9415b6b73 2019-11-25T09:14:33.344 INFO:teuthology.orchestra.run.smithi153.stdout:last_changed 2019-11-25T09:14:33.344342+0000 2019-11-25T09:14:33.344 INFO:teuthology.orchestra.run.smithi153.stdout:created 2019-11-25T09:14:33.344342+0000 2019-11-25T09:14:33.344 INFO:teuthology.orchestra.run.smithi153.stdout:min_mon_release 0 (unknown) 2019-11-25T09:14:33.344 INFO:teuthology.orchestra.run.smithi153.stdout:0: [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] mon.a 2019-11-25T09:14:33.344 INFO:teuthology.orchestra.run.smithi153.stdout:1: [v2:172.21.15.81:3300/0,v1:172.21.15.81:6789/0] mon.b 2019-11-25T09:14:33.345 INFO:teuthology.orchestra.run.smithi153.stdout:2: [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] mon.c 2019-11-25T09:14:33.345 INFO:teuthology.orchestra.run.smithi153.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (3 monitors) 2019-11-25T09:14:33.346 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 89933446-017a-4383-871e-aaf9415b6b73... 2019-11-25T09:14:33.347 INFO:teuthology.orchestra.run.smithi081:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-11-25T09:14:33.352 INFO:teuthology.orchestra.run.smithi153:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo python -c 'import shutil, sys; shutil.copyfileobj(sys.stdin, file(sys.argv[1], "wb"))' /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf 2019-11-25T09:14:33.559 INFO:tasks.ceph:Creating admin key on mon.a... 2019-11-25T09:14:33.559 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2019-11-25T09:14:33.708 INFO:tasks.ceph:Copying monmap to all nodes... 2019-11-25T09:14:33.807 DEBUG:teuthology.orchestra.remote:smithi153:/etc/ceph/ceph.keyring is 206B 2019-11-25T09:14:33.944 DEBUG:teuthology.orchestra.remote:smithi153:/home/ubuntu/cephtest/ceph.monmap is 379B 2019-11-25T09:14:34.008 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:14:34.008 INFO:teuthology.orchestra.run.smithi153:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-11-25T09:14:34.112 INFO:teuthology.orchestra.run.smithi153:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:34.171 INFO:tasks.ceph:Sending monmap to node ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:14:34.172 INFO:teuthology.orchestra.run.smithi081:> sudo sh -c 'cat > /etc/ceph/ceph.keyring' && sudo chmod 0644 /etc/ceph/ceph.keyring 2019-11-25T09:14:34.322 INFO:teuthology.orchestra.run.smithi081:> cat > /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:34.391 INFO:tasks.ceph:Setting up mon nodes... 2019-11-25T09:14:34.391 INFO:tasks.ceph:Setting up mgr nodes... 2019-11-25T09:14:34.392 INFO:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mgr/ceph-y && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.y /var/lib/ceph/mgr/ceph-y/keyring 2019-11-25T09:14:34.564 INFO:teuthology.orchestra.run.smithi153.stdout:creating /var/lib/ceph/mgr/ceph-y/keyring 2019-11-25T09:14:34.569 INFO:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2019-11-25T09:14:34.774 INFO:teuthology.orchestra.run.smithi081.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2019-11-25T09:14:34.780 INFO:tasks.ceph:Setting up mds nodes... 2019-11-25T09:14:34.780 INFO:tasks.ceph_client:Setting up client nodes... 2019-11-25T09:14:34.780 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2019-11-25T09:14:34.931 INFO:teuthology.orchestra.run.smithi153.stdout:creating /etc/ceph/ceph.client.0.keyring 2019-11-25T09:14:34.963 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2019-11-25T09:14:35.121 INFO:teuthology.orchestra.run.smithi081.stdout:creating /etc/ceph/ceph.client.1.keyring 2019-11-25T09:14:35.153 INFO:tasks.ceph:Running mkfs on osd nodes... 2019-11-25T09:14:35.153 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@smithi153.front.sepia.ceph.com'): {'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@smithi081.front.sepia.ceph.com'): {'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'}} 2019-11-25T09:14:35.153 INFO:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2019-11-25T09:14:35.273 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:35.273 INFO:tasks.ceph:{} 2019-11-25T09:14:35.273 INFO:tasks.ceph:osd.0 2019-11-25T09:14:35.273 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:14:35.274 INFO:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-11-25T09:14:35.882 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-11-25T09:14:35.882 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-25T09:14:35.882 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=0, sparse=0 2019-11-25T09:14:35.882 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-25T09:14:35.882 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2019-11-25T09:14:35.883 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-25T09:14:35.883 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-25T09:14:35.883 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-25T09:14:35.883 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-25T09:14:35.884 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2019-11-25T09:14:35.885 INFO:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-0 2019-11-25T09:14:36.063 INFO:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2019-11-25T09:14:36.170 INFO:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2019-11-25T09:14:36.248 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:36.248 INFO:tasks.ceph:{} 2019-11-25T09:14:36.248 INFO:tasks.ceph:osd.1 2019-11-25T09:14:36.249 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:14:36.249 INFO:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-11-25T09:14:36.847 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-11-25T09:14:36.848 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-25T09:14:36.848 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=0, sparse=0 2019-11-25T09:14:36.848 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-25T09:14:36.848 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2019-11-25T09:14:36.848 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-25T09:14:36.848 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-25T09:14:36.849 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-25T09:14:36.849 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-25T09:14:36.850 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2019-11-25T09:14:36.850 INFO:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-1 2019-11-25T09:14:36.995 INFO:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2019-11-25T09:14:37.116 INFO:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2019-11-25T09:14:37.227 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:37.227 INFO:tasks.ceph:{} 2019-11-25T09:14:37.227 INFO:tasks.ceph:osd.2 2019-11-25T09:14:37.227 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:14:37.227 INFO:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-11-25T09:14:37.846 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-11-25T09:14:37.846 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-25T09:14:37.846 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=0, sparse=0 2019-11-25T09:14:37.846 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-25T09:14:37.846 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2019-11-25T09:14:37.847 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-25T09:14:37.847 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-25T09:14:37.847 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-25T09:14:37.847 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-25T09:14:37.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2019-11-25T09:14:37.849 INFO:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-2 2019-11-25T09:14:38.016 INFO:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2019-11-25T09:14:38.132 INFO:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2019-11-25T09:14:38.222 INFO:tasks.ceph:{'osd.1': '/dev/vg_nvme/lv_1', 'osd.0': '/dev/vg_nvme/lv_4', 'osd.3': '/dev/vg_nvme/lv_3', 'osd.2': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:38.223 INFO:tasks.ceph:{} 2019-11-25T09:14:38.223 INFO:tasks.ceph:osd.3 2019-11-25T09:14:38.223 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:14:38.223 INFO:teuthology.orchestra.run.smithi153:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-11-25T09:14:38.805 INFO:teuthology.orchestra.run.smithi153.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-11-25T09:14:38.805 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-25T09:14:38.805 INFO:teuthology.orchestra.run.smithi153.stdout: = crc=1 finobt=0, sparse=0 2019-11-25T09:14:38.805 INFO:teuthology.orchestra.run.smithi153.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-25T09:14:38.805 INFO:teuthology.orchestra.run.smithi153.stdout: = sunit=0 swidth=0 blks 2019-11-25T09:14:38.806 INFO:teuthology.orchestra.run.smithi153.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-25T09:14:38.806 INFO:teuthology.orchestra.run.smithi153.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-25T09:14:38.806 INFO:teuthology.orchestra.run.smithi153.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-25T09:14:38.806 INFO:teuthology.orchestra.run.smithi153.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-25T09:14:38.808 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi153.front.sepia.ceph.com -o noatime 2019-11-25T09:14:38.808 INFO:teuthology.orchestra.run.smithi153:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-3 2019-11-25T09:14:38.956 INFO:teuthology.orchestra.run.smithi153:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2019-11-25T09:14:39.052 INFO:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:39.203 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.202+0000 7f86f0354b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-0/keyring: can't open /var/lib/ceph/osd/ceph-0/keyring: (2) No such file or directory 2019-11-25T09:14:39.205 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.205+0000 7f86f0354b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-0/keyring 2019-11-25T09:14:39.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.246+0000 7f86f0354b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:39.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.246+0000 7f86f0354b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-25T09:14:39.501 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.501+0000 7f86f0354b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:39.504 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.504+0000 7f86f0354b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:39.504 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.504+0000 7f86f0354b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:39.531 INFO:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2019-11-25T09:14:39.641 INFO:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:39.789 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.787+0000 7f49599cab40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-1/keyring: can't open /var/lib/ceph/osd/ceph-1/keyring: (2) No such file or directory 2019-11-25T09:14:39.790 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.790+0000 7f49599cab40 -1 created new key in keyring /var/lib/ceph/osd/ceph-1/keyring 2019-11-25T09:14:39.827 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.827+0000 7f49599cab40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:39.828 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:39.828+0000 7f49599cab40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-25T09:14:40.058 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.049+0000 7f49599cab40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:40.058 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.049+0000 7f49599cab40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:40.067 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.049+0000 7f49599cab40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:40.068 INFO:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2019-11-25T09:14:40.203 INFO:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:40.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.337+0000 7f3894418b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-2/keyring: can't open /var/lib/ceph/osd/ceph-2/keyring: (2) No such file or directory 2019-11-25T09:14:40.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.337+0000 7f3894418b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-2/keyring 2019-11-25T09:14:40.353 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.353+0000 7f3894418b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:40.353 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.353+0000 7f3894418b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-25T09:14:40.498 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.498+0000 7f3894418b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:40.498 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.498+0000 7f3894418b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:40.498 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.498+0000 7f3894418b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:40.511 INFO:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2019-11-25T09:14:40.599 INFO:teuthology.orchestra.run.smithi153:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:40.706 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.706+0000 7f3d402c4b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-3/keyring: can't open /var/lib/ceph/osd/ceph-3/keyring: (2) No such file or directory 2019-11-25T09:14:40.707 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.707+0000 7f3d402c4b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-3/keyring 2019-11-25T09:14:40.718 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.719+0000 7f3d402c4b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:40.718 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.719+0000 7f3d402c4b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-25T09:14:40.867 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.867+0000 7f3d402c4b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:40.867 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.867+0000 7f3d402c4b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:40.868 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:14:40.867+0000 7f3d402c4b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:40.881 INFO:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2019-11-25T09:14:40.977 INFO:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-4 2019-11-25T09:14:41.098 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:41.098 INFO:tasks.ceph:{} 2019-11-25T09:14:41.099 INFO:tasks.ceph:osd.4 2019-11-25T09:14:41.099 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:14:41.099 INFO:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_4 2019-11-25T09:14:41.685 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_4 isize=2048 agcount=4, agsize=5859072 blks 2019-11-25T09:14:41.685 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-25T09:14:41.686 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=0, sparse=0 2019-11-25T09:14:41.686 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-25T09:14:41.686 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2019-11-25T09:14:41.686 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-25T09:14:41.686 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-25T09:14:41.686 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-25T09:14:41.686 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-25T09:14:41.689 INFO:tasks.ceph:mount /dev/vg_nvme/lv_4 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2019-11-25T09:14:41.690 INFO:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_4 /var/lib/ceph/osd/ceph-4 2019-11-25T09:14:41.889 INFO:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-4 2019-11-25T09:14:42.013 INFO:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-5 2019-11-25T09:14:42.113 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:42.113 INFO:tasks.ceph:{} 2019-11-25T09:14:42.114 INFO:tasks.ceph:osd.5 2019-11-25T09:14:42.114 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:14:42.114 INFO:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2019-11-25T09:14:42.692 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=5859072 blks 2019-11-25T09:14:42.692 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-25T09:14:42.692 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=0, sparse=0 2019-11-25T09:14:42.693 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-25T09:14:42.693 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2019-11-25T09:14:42.693 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-25T09:14:42.693 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-25T09:14:42.693 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-25T09:14:42.694 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-25T09:14:42.694 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2019-11-25T09:14:42.695 INFO:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-5 2019-11-25T09:14:42.845 INFO:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-5 2019-11-25T09:14:42.924 INFO:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-6 2019-11-25T09:14:42.993 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:42.993 INFO:tasks.ceph:{} 2019-11-25T09:14:42.993 INFO:tasks.ceph:osd.6 2019-11-25T09:14:42.993 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:14:42.994 INFO:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2019-11-25T09:14:43.556 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=5859072 blks 2019-11-25T09:14:43.556 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-25T09:14:43.556 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=0, sparse=0 2019-11-25T09:14:43.556 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-25T09:14:43.557 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2019-11-25T09:14:43.557 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-25T09:14:43.557 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-25T09:14:43.557 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-25T09:14:43.557 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-25T09:14:43.558 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2019-11-25T09:14:43.559 INFO:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-6 2019-11-25T09:14:43.735 INFO:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-6 2019-11-25T09:14:43.853 INFO:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/osd/ceph-7 2019-11-25T09:14:43.942 INFO:tasks.ceph:{'osd.5': '/dev/vg_nvme/lv_1', 'osd.4': '/dev/vg_nvme/lv_4', 'osd.7': '/dev/vg_nvme/lv_3', 'osd.6': '/dev/vg_nvme/lv_2'} 2019-11-25T09:14:43.943 INFO:tasks.ceph:{} 2019-11-25T09:14:43.943 INFO:tasks.ceph:osd.7 2019-11-25T09:14:43.943 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:14:43.943 INFO:teuthology.orchestra.run.smithi081:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2019-11-25T09:14:44.482 INFO:teuthology.orchestra.run.smithi081.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=5859072 blks 2019-11-25T09:14:44.482 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 attr=2, projid32bit=1 2019-11-25T09:14:44.482 INFO:teuthology.orchestra.run.smithi081.stdout: = crc=1 finobt=0, sparse=0 2019-11-25T09:14:44.483 INFO:teuthology.orchestra.run.smithi081.stdout:data = bsize=4096 blocks=23436288, imaxpct=25 2019-11-25T09:14:44.483 INFO:teuthology.orchestra.run.smithi081.stdout: = sunit=0 swidth=0 blks 2019-11-25T09:14:44.483 INFO:teuthology.orchestra.run.smithi081.stdout:naming =version 2 bsize=4096 ascii-ci=0 ftype=1 2019-11-25T09:14:44.483 INFO:teuthology.orchestra.run.smithi081.stdout:log =internal log bsize=4096 blocks=11443, version=2 2019-11-25T09:14:44.483 INFO:teuthology.orchestra.run.smithi081.stdout: = sectsz=512 sunit=0 blks, lazy-count=1 2019-11-25T09:14:44.483 INFO:teuthology.orchestra.run.smithi081.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2019-11-25T09:14:44.485 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@smithi081.front.sepia.ceph.com -o noatime 2019-11-25T09:14:44.485 INFO:teuthology.orchestra.run.smithi081:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-7 2019-11-25T09:14:44.610 INFO:teuthology.orchestra.run.smithi081:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-7 2019-11-25T09:14:44.719 INFO:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 4 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:44.860 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:44.860+0000 7fc0c0559b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-4/keyring: can't open /var/lib/ceph/osd/ceph-4/keyring: (2) No such file or directory 2019-11-25T09:14:44.861 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:44.860+0000 7fc0c0559b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-4/keyring 2019-11-25T09:14:44.872 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:44.872+0000 7fc0c0559b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:44.872 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:44.872+0000 7fc0c0559b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-25T09:14:45.020 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.020+0000 7fc0c0559b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:45.021 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.020+0000 7fc0c0559b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:45.021 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.020+0000 7fc0c0559b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:45.032 INFO:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-4 2019-11-25T09:14:45.125 INFO:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 5 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:45.238 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.237+0000 7f4b0700db40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-5/keyring: can't open /var/lib/ceph/osd/ceph-5/keyring: (2) No such file or directory 2019-11-25T09:14:45.240 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.238+0000 7f4b0700db40 -1 created new key in keyring /var/lib/ceph/osd/ceph-5/keyring 2019-11-25T09:14:45.253 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.253+0000 7f4b0700db40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:45.253 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.253+0000 7f4b0700db40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-25T09:14:45.394 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.394+0000 7f4b0700db40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:45.394 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.394+0000 7f4b0700db40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:45.394 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.394+0000 7f4b0700db40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:45.405 INFO:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-5 2019-11-25T09:14:45.480 INFO:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 6 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:45.572 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.571+0000 7f5fbb6e8b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-6/keyring: can't open /var/lib/ceph/osd/ceph-6/keyring: (2) No such file or directory 2019-11-25T09:14:45.572 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.572+0000 7f5fbb6e8b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-6/keyring 2019-11-25T09:14:45.582 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.582+0000 7f5fbb6e8b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:45.582 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.582+0000 7f5fbb6e8b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-25T09:14:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.719+0000 7f5fbb6e8b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.719+0000 7f5fbb6e8b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:45.719 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.719+0000 7f5fbb6e8b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:45.732 INFO:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-6 2019-11-25T09:14:45.822 INFO:teuthology.orchestra.run.smithi081:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 7 --monmap /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:45.924 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.923+0000 7f3d12bf4b40 -1 auth: error reading file: /var/lib/ceph/osd/ceph-7/keyring: can't open /var/lib/ceph/osd/ceph-7/keyring: (2) No such file or directory 2019-11-25T09:14:45.924 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.924+0000 7f3d12bf4b40 -1 created new key in keyring /var/lib/ceph/osd/ceph-7/keyring 2019-11-25T09:14:45.932 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.932+0000 7f3d12bf4b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:45.933 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:45.932+0000 7f3d12bf4b40 -1 journal FileJournal::_open_file : zeroing journal 2019-11-25T09:14:46.069 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:46.069+0000 7f3d12bf4b40 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:14:46.069 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:46.069+0000 7f3d12bf4b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:46.070 INFO:teuthology.orchestra.run.smithi081.stderr:2019-11-25T09:14:46.069+0000 7f3d12bf4b40 -1 journal do_read_entry(4096): bad header magic 2019-11-25T09:14:46.081 INFO:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-7 2019-11-25T09:14:46.151 INFO:tasks.ceph:Reading keys from all nodes... 2019-11-25T09:14:46.152 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:14:46.260 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.tPeb55DPpX 2019-11-25T09:14:46.261 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/mgr/ceph-y/keyring /tmp/tmp.tPeb55DPpX 2019-11-25T09:14:46.379 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.tPeb55DPpX 2019-11-25T09:14:46.570 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.tPeb55DPpX is 56B 2019-11-25T09:14:46.645 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.tPeb55DPpX 2019-11-25T09:14:46.714 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:14:46.791 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.TKVTD2ya2p 2019-11-25T09:14:46.791 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/osd/ceph-0/keyring /tmp/tmp.TKVTD2ya2p 2019-11-25T09:14:46.876 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.TKVTD2ya2p 2019-11-25T09:14:47.010 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.TKVTD2ya2p is 56B 2019-11-25T09:14:47.070 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.TKVTD2ya2p 2019-11-25T09:14:47.117 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:14:47.257 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.gLoUut7uym 2019-11-25T09:14:47.258 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/osd/ceph-1/keyring /tmp/tmp.gLoUut7uym 2019-11-25T09:14:47.380 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.gLoUut7uym 2019-11-25T09:14:47.625 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.gLoUut7uym is 56B 2019-11-25T09:14:47.723 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.gLoUut7uym 2019-11-25T09:14:47.801 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:14:47.884 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.JTIkoW8ZDa 2019-11-25T09:14:47.955 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/osd/ceph-2/keyring /tmp/tmp.JTIkoW8ZDa 2019-11-25T09:14:48.082 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.JTIkoW8ZDa 2019-11-25T09:14:48.264 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.JTIkoW8ZDa is 56B 2019-11-25T09:14:48.346 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.JTIkoW8ZDa 2019-11-25T09:14:48.413 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:14:48.559 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.XUTAnxkpfZ 2019-11-25T09:14:48.560 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/osd/ceph-3/keyring /tmp/tmp.XUTAnxkpfZ 2019-11-25T09:14:48.658 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.XUTAnxkpfZ 2019-11-25T09:14:48.811 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.XUTAnxkpfZ is 56B 2019-11-25T09:14:48.880 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.XUTAnxkpfZ 2019-11-25T09:14:48.946 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:14:49.034 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.vaJVIwT8x2 2019-11-25T09:14:49.035 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/mgr/ceph-x/keyring /tmp/tmp.vaJVIwT8x2 2019-11-25T09:14:49.130 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.vaJVIwT8x2 2019-11-25T09:14:49.271 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.vaJVIwT8x2 is 56B 2019-11-25T09:14:49.328 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.vaJVIwT8x2 2019-11-25T09:14:49.384 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:14:49.514 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.30iupt2ot4 2019-11-25T09:14:49.515 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/osd/ceph-4/keyring /tmp/tmp.30iupt2ot4 2019-11-25T09:14:49.623 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.30iupt2ot4 2019-11-25T09:14:49.807 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.30iupt2ot4 is 56B 2019-11-25T09:14:49.879 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.30iupt2ot4 2019-11-25T09:14:49.940 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:14:50.000 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.15c2MDHAV5 2019-11-25T09:14:50.000 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/osd/ceph-5/keyring /tmp/tmp.15c2MDHAV5 2019-11-25T09:14:50.081 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.15c2MDHAV5 2019-11-25T09:14:50.214 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.15c2MDHAV5 is 56B 2019-11-25T09:14:50.293 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.15c2MDHAV5 2019-11-25T09:14:50.390 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:14:50.481 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.zZpULuqai6 2019-11-25T09:14:50.481 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/osd/ceph-6/keyring /tmp/tmp.zZpULuqai6 2019-11-25T09:14:50.589 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.zZpULuqai6 2019-11-25T09:14:50.789 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.zZpULuqai6 is 56B 2019-11-25T09:14:50.911 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.zZpULuqai6 2019-11-25T09:14:51.005 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:14:51.084 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.17bJWXh521 2019-11-25T09:14:51.085 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/osd/ceph-7/keyring /tmp/tmp.17bJWXh521 2019-11-25T09:14:51.176 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.17bJWXh521 2019-11-25T09:14:51.323 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.17bJWXh521 is 56B 2019-11-25T09:14:51.385 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.17bJWXh521 2019-11-25T09:14:51.535 DEBUG:teuthology.orchestra.remote:smithi153:/etc/ceph/ceph.client.0.keyring is 59B 2019-11-25T09:14:51.723 DEBUG:teuthology.orchestra.remote:smithi081:/etc/ceph/ceph.client.1.keyring is 59B 2019-11-25T09:14:51.812 INFO:tasks.ceph:Adding keys to all mons... 2019-11-25T09:14:51.812 INFO:teuthology.orchestra.run.smithi081:> sudo tee -a /etc/ceph/ceph.keyring 2019-11-25T09:14:51.817 INFO:teuthology.orchestra.run.smithi153:> sudo tee -a /etc/ceph/ceph.keyring 2019-11-25T09:14:51.916 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.y] 2019-11-25T09:14:51.917 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQB6m9tdUlOvIRAATb9wkE1WgB/6yABE9D8N2g== 2019-11-25T09:14:51.917 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.0] 2019-11-25T09:14:51.917 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQB/m9tdcAccDBAAgFP0duxLYT9RN9LQJFlA4w== 2019-11-25T09:14:51.917 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.1] 2019-11-25T09:14:51.917 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQB/m9tdZwb4LhAAoAEArJtW4yyi/tNihG8Dug== 2019-11-25T09:14:51.917 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.2] 2019-11-25T09:14:51.918 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCAm9tdBdcpFBAA5PdaK10R5yLCtwXL0t0fOQ== 2019-11-25T09:14:51.918 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.3] 2019-11-25T09:14:51.918 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCAm9tdO1UtKhAAkcnv41Ov18GYFQ3V9XiPdA== 2019-11-25T09:14:51.918 INFO:teuthology.orchestra.run.smithi081.stdout:[mgr.x] 2019-11-25T09:14:51.918 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQB6m9td3TI6LhAAP/GZsa56Lp7xWqw86LwVhA== 2019-11-25T09:14:51.918 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.4] 2019-11-25T09:14:51.918 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCEm9tdslhYMxAAXZ9P8knbbFZV1od9Ht8bQQ== 2019-11-25T09:14:51.918 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.5] 2019-11-25T09:14:51.919 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCFm9tdicVDDhAAM3uwK3rp86QP0qPIKWuXyQ== 2019-11-25T09:14:51.919 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.6] 2019-11-25T09:14:51.919 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCFm9tdFYwqIhAAkBBvGWy2B6A2ttCNu37jlA== 2019-11-25T09:14:51.919 INFO:teuthology.orchestra.run.smithi081.stdout:[osd.7] 2019-11-25T09:14:51.919 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQCFm9tdCrsiNxAATZqwimNIhmBAr/6R+2TfAQ== 2019-11-25T09:14:51.920 INFO:teuthology.orchestra.run.smithi081.stdout:[client.0] 2019-11-25T09:14:51.920 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQB6m9tdva+RNxAA65uvN7+kO8rMNPO1loc2YA== 2019-11-25T09:14:51.920 INFO:teuthology.orchestra.run.smithi081.stdout:[client.1] 2019-11-25T09:14:51.920 INFO:teuthology.orchestra.run.smithi081.stdout: key = AQB7m9tdCn5HBxAA/vebfRSDQvqxNqiy/1IAXg== 2019-11-25T09:14:51.936 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.y] 2019-11-25T09:14:51.937 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQB6m9tdUlOvIRAATb9wkE1WgB/6yABE9D8N2g== 2019-11-25T09:14:51.937 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.0] 2019-11-25T09:14:51.937 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQB/m9tdcAccDBAAgFP0duxLYT9RN9LQJFlA4w== 2019-11-25T09:14:51.937 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.1] 2019-11-25T09:14:51.937 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQB/m9tdZwb4LhAAoAEArJtW4yyi/tNihG8Dug== 2019-11-25T09:14:51.937 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.2] 2019-11-25T09:14:51.937 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCAm9tdBdcpFBAA5PdaK10R5yLCtwXL0t0fOQ== 2019-11-25T09:14:51.938 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.3] 2019-11-25T09:14:51.938 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCAm9tdO1UtKhAAkcnv41Ov18GYFQ3V9XiPdA== 2019-11-25T09:14:51.938 INFO:teuthology.orchestra.run.smithi153.stdout:[mgr.x] 2019-11-25T09:14:51.938 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQB6m9td3TI6LhAAP/GZsa56Lp7xWqw86LwVhA== 2019-11-25T09:14:51.938 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.4] 2019-11-25T09:14:51.938 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCEm9tdslhYMxAAXZ9P8knbbFZV1od9Ht8bQQ== 2019-11-25T09:14:51.938 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.5] 2019-11-25T09:14:51.939 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCFm9tdicVDDhAAM3uwK3rp86QP0qPIKWuXyQ== 2019-11-25T09:14:51.939 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.6] 2019-11-25T09:14:51.939 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCFm9tdFYwqIhAAkBBvGWy2B6A2ttCNu37jlA== 2019-11-25T09:14:51.939 INFO:teuthology.orchestra.run.smithi153.stdout:[osd.7] 2019-11-25T09:14:51.939 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQCFm9tdCrsiNxAATZqwimNIhmBAr/6R+2TfAQ== 2019-11-25T09:14:51.940 INFO:teuthology.orchestra.run.smithi153.stdout:[client.0] 2019-11-25T09:14:51.940 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQB6m9tdva+RNxAA65uvN7+kO8rMNPO1loc2YA== 2019-11-25T09:14:51.941 INFO:teuthology.orchestra.run.smithi153.stdout:[client.1] 2019-11-25T09:14:51.941 INFO:teuthology.orchestra.run.smithi153.stdout: key = AQB7m9tdCn5HBxAA/vebfRSDQvqxNqiy/1IAXg== 2019-11-25T09:14:51.942 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-11-25T09:14:51.946 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.y --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-11-25T09:14:52.115 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.120 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.293 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.298 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.460 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.464 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.638 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.642 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.815 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-11-25T09:14:52.820 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mds 'allow *' --cap osd 'allow *' --cap mon 'allow profile mgr' 2019-11-25T09:14:52.962 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:52.967 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.4 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:53.132 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:53.137 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.5 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:53.326 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:53.331 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.6 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:53.488 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:53.493 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.7 --cap mgr 'allow *' --cap osd 'allow *' --cap mon 'allow *' 2019-11-25T09:14:53.657 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-11-25T09:14:53.663 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-11-25T09:14:53.835 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-11-25T09:14:53.840 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mds allow --cap osd 'allow rwx' --cap mgr 'allow r' --cap mon 'allow rw' 2019-11-25T09:14:54.013 INFO:tasks.ceph:Running mkfs on mon nodes... 2019-11-25T09:14:54.013 INFO:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2019-11-25T09:14:54.134 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-11-25T09:14:54.305 INFO:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2019-11-25T09:14:54.402 INFO:teuthology.orchestra.run.smithi153:> sudo mkdir -p /var/lib/ceph/mon/ceph-c 2019-11-25T09:14:54.493 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i c --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-11-25T09:14:54.622 INFO:teuthology.orchestra.run.smithi153:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-c 2019-11-25T09:14:54.711 INFO:teuthology.orchestra.run.smithi081:> sudo mkdir -p /var/lib/ceph/mon/ceph-b 2019-11-25T09:14:54.831 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i b --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2019-11-25T09:14:55.024 INFO:teuthology.orchestra.run.smithi081:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-b 2019-11-25T09:14:55.128 INFO:teuthology.orchestra.run.smithi081:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:55.132 INFO:teuthology.orchestra.run.smithi153:> rm -- /home/ubuntu/cephtest/ceph.monmap 2019-11-25T09:14:55.212 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2019-11-25T09:14:55.213 DEBUG:teuthology.misc:running mon.a under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.a.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-11-25T09:14:55.213 INFO:tasks.ceph.mon.a:Restarting daemon 2019-11-25T09:14:55.213 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.a.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i a 2019-11-25T09:14:55.218 INFO:tasks.ceph.mon.a:Started 2019-11-25T09:14:55.218 DEBUG:teuthology.misc:running mon.c under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.c.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-11-25T09:14:55.218 INFO:tasks.ceph.mon.c:Restarting daemon 2019-11-25T09:14:55.219 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.c.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i c 2019-11-25T09:14:55.223 INFO:tasks.ceph.mon.c:Started 2019-11-25T09:14:55.223 DEBUG:teuthology.misc:running mon.b under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/mon.b.log', '--time-stamp=yes', '--tool=memcheck', '--leak-check=full', '--show-reachable=yes'] 2019-11-25T09:14:55.223 INFO:tasks.ceph.mon.b:Restarting daemon 2019-11-25T09:14:55.223 INFO:teuthology.orchestra.run.smithi081:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/mon.b.log --time-stamp=yes --tool=memcheck --leak-check=full --show-reachable=yes ceph-mon -f --cluster ceph -i b 2019-11-25T09:14:55.229 INFO:tasks.ceph.mon.b:Started 2019-11-25T09:14:55.229 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2019-11-25T09:14:55.229 INFO:tasks.ceph.mgr.y:Restarting daemon 2019-11-25T09:14:55.229 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i y 2019-11-25T09:14:55.234 INFO:tasks.ceph.mgr.y:Started 2019-11-25T09:14:55.235 INFO:tasks.ceph.mgr.x:Restarting daemon 2019-11-25T09:14:55.236 INFO:teuthology.orchestra.run.smithi081:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term ceph-mgr -f --cluster ceph -i x 2019-11-25T09:14:55.242 INFO:tasks.ceph.mgr.x:Started 2019-11-25T09:14:55.242 INFO:tasks.ceph:Setting crush tunables to default 2019-11-25T09:14:55.242 INFO:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd crush tunables default 2019-11-25T09:14:59.200 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:00:03.809 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:14:59.235 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:00:03.821 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:14:59.358 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:00:03.967 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:06.063 INFO:teuthology.orchestra.run.smithi153.stderr:adjusted tunables profile to default 2019-11-25T09:15:06.077 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2019-11-25T09:15:06.078 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:15:06.134 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.bZ5piXJrNg 2019-11-25T09:15:06.134 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/osd/ceph-0/fsid /tmp/tmp.bZ5piXJrNg 2019-11-25T09:15:06.192 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.bZ5piXJrNg 2019-11-25T09:15:06.307 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.bZ5piXJrNg is 37B 2019-11-25T09:15:06.360 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.bZ5piXJrNg 2019-11-25T09:15:06.403 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:15:06.518 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.IY4zukvcZi 2019-11-25T09:15:06.519 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/osd/ceph-1/fsid /tmp/tmp.IY4zukvcZi 2019-11-25T09:15:06.585 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.IY4zukvcZi 2019-11-25T09:15:06.710 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.IY4zukvcZi is 37B 2019-11-25T09:15:06.761 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.IY4zukvcZi 2019-11-25T09:15:06.806 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:15:06.918 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.TiIaqe9x0v 2019-11-25T09:15:06.918 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/osd/ceph-2/fsid /tmp/tmp.TiIaqe9x0v 2019-11-25T09:15:06.972 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.TiIaqe9x0v 2019-11-25T09:15:07.070 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.TiIaqe9x0v is 37B 2019-11-25T09:15:07.119 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.TiIaqe9x0v 2019-11-25T09:15:07.161 INFO:teuthology.orchestra.run.smithi153:> mktemp 2019-11-25T09:15:07.292 INFO:teuthology.orchestra.run.smithi153.stdout:/tmp/tmp.q4Dp7HfZzE 2019-11-25T09:15:07.293 INFO:teuthology.orchestra.run.smithi153:> sudo cp /var/lib/ceph/osd/ceph-3/fsid /tmp/tmp.q4Dp7HfZzE 2019-11-25T09:15:07.361 INFO:teuthology.orchestra.run.smithi153:> sudo chmod 0666 /tmp/tmp.q4Dp7HfZzE 2019-11-25T09:15:07.466 DEBUG:teuthology.orchestra.remote:smithi153:/tmp/tmp.q4Dp7HfZzE is 37B 2019-11-25T09:15:07.515 INFO:teuthology.orchestra.run.smithi153:> rm -fr /tmp/tmp.q4Dp7HfZzE 2019-11-25T09:15:07.557 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:15:07.649 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.tPHSyxrkZH 2019-11-25T09:15:07.649 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/osd/ceph-4/fsid /tmp/tmp.tPHSyxrkZH 2019-11-25T09:15:07.751 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.tPHSyxrkZH 2019-11-25T09:15:07.887 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.tPHSyxrkZH is 37B 2019-11-25T09:15:07.946 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.tPHSyxrkZH 2019-11-25T09:15:07.998 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:15:08.112 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.xoK8Bur95n 2019-11-25T09:15:08.112 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/osd/ceph-5/fsid /tmp/tmp.xoK8Bur95n 2019-11-25T09:15:08.172 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.xoK8Bur95n 2019-11-25T09:15:08.277 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.xoK8Bur95n is 37B 2019-11-25T09:15:08.329 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.xoK8Bur95n 2019-11-25T09:15:08.375 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:15:08.502 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.Q8YKiLywRe 2019-11-25T09:15:08.502 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/osd/ceph-6/fsid /tmp/tmp.Q8YKiLywRe 2019-11-25T09:15:08.565 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.Q8YKiLywRe 2019-11-25T09:15:08.679 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.Q8YKiLywRe is 37B 2019-11-25T09:15:08.730 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.Q8YKiLywRe 2019-11-25T09:15:08.774 INFO:teuthology.orchestra.run.smithi081:> mktemp 2019-11-25T09:15:08.915 INFO:teuthology.orchestra.run.smithi081.stdout:/tmp/tmp.HOkCtreM8T 2019-11-25T09:15:08.916 INFO:teuthology.orchestra.run.smithi081:> sudo cp /var/lib/ceph/osd/ceph-7/fsid /tmp/tmp.HOkCtreM8T 2019-11-25T09:15:09.013 INFO:teuthology.orchestra.run.smithi081:> sudo chmod 0666 /tmp/tmp.HOkCtreM8T 2019-11-25T09:15:09.143 DEBUG:teuthology.orchestra.remote:smithi081:/tmp/tmp.HOkCtreM8T is 37B 2019-11-25T09:15:09.207 INFO:teuthology.orchestra.run.smithi081:> rm -fr /tmp/tmp.HOkCtreM8T 2019-11-25T09:15:09.254 INFO:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 45513261-33b5-44de-a471-c28df1efe738 0 2019-11-25T09:15:10.567 INFO:teuthology.orchestra.run.smithi081.stdout:0 2019-11-25T09:15:10.586 INFO:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 13974267-191f-4964-8d94-59b9df2b785e 1 2019-11-25T09:15:11.482 INFO:teuthology.orchestra.run.smithi081.stdout:1 2019-11-25T09:15:11.500 INFO:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new af214156-9298-45bd-a496-38c087d74f40 2 2019-11-25T09:15:12.563 INFO:teuthology.orchestra.run.smithi081.stdout:2 2019-11-25T09:15:12.585 INFO:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new fa7627c3-b1e2-4470-896e-c7ae3e921848 3 2019-11-25T09:15:13.641 INFO:teuthology.orchestra.run.smithi081.stdout:3 2019-11-25T09:15:13.661 INFO:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 180d6e76-1a57-40c1-b27b-33b9b2c22f5d 4 2019-11-25T09:15:14.554 INFO:teuthology.orchestra.run.smithi081.stdout:4 2019-11-25T09:15:14.578 INFO:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new f43c89af-c05e-4f72-959b-e71ac857cf5e 5 2019-11-25T09:15:15.683 INFO:teuthology.orchestra.run.smithi081.stdout:5 2019-11-25T09:15:15.718 INFO:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new d92af41b-8ff9-4727-91ea-1a4c924fc90e 6 2019-11-25T09:15:16.669 INFO:teuthology.orchestra.run.smithi081.stdout:6 2019-11-25T09:15:16.685 INFO:teuthology.orchestra.run.smithi081:> sudo ceph --cluster ceph osd new 7e9cedf9-ab32-48f1-8afc-14140c898611 7 2019-11-25T09:15:18.070 INFO:teuthology.orchestra.run.smithi081.stdout:7 2019-11-25T09:15:18.092 DEBUG:teuthology.misc:running osd.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.0.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-25T09:15:18.092 INFO:tasks.ceph.osd.0:Restarting daemon 2019-11-25T09:15:18.093 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.0.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 0 2019-11-25T09:15:18.097 INFO:tasks.ceph.osd.0:Started 2019-11-25T09:15:18.097 DEBUG:teuthology.misc:running osd.1 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.1.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-25T09:15:18.098 INFO:tasks.ceph.osd.1:Restarting daemon 2019-11-25T09:15:18.098 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.1.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 1 2019-11-25T09:15:18.102 INFO:tasks.ceph.osd.1:Started 2019-11-25T09:15:18.103 DEBUG:teuthology.misc:running osd.2 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.2.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-25T09:15:18.103 INFO:tasks.ceph.osd.2:Restarting daemon 2019-11-25T09:15:18.103 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.2.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 2 2019-11-25T09:15:18.108 INFO:tasks.ceph.osd.2:Started 2019-11-25T09:15:18.108 DEBUG:teuthology.misc:running osd.3 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.3.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-25T09:15:18.108 INFO:tasks.ceph.osd.3:Restarting daemon 2019-11-25T09:15:18.108 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.3.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 3 2019-11-25T09:15:18.113 INFO:tasks.ceph.osd.3:Started 2019-11-25T09:15:18.113 DEBUG:teuthology.misc:running osd.4 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.4.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-25T09:15:18.114 INFO:tasks.ceph.osd.4:Restarting daemon 2019-11-25T09:15:18.114 INFO:teuthology.orchestra.run.smithi081:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.4.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 4 2019-11-25T09:15:18.121 INFO:tasks.ceph.osd.4:Started 2019-11-25T09:15:18.121 DEBUG:teuthology.misc:running osd.5 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.5.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-25T09:15:18.122 INFO:tasks.ceph.osd.5:Restarting daemon 2019-11-25T09:15:18.122 INFO:teuthology.orchestra.run.smithi081:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.5.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 5 2019-11-25T09:15:18.126 INFO:tasks.ceph.osd.5:Started 2019-11-25T09:15:18.126 DEBUG:teuthology.misc:running osd.6 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.6.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-25T09:15:18.126 INFO:tasks.ceph.osd.6:Restarting daemon 2019-11-25T09:15:18.127 INFO:teuthology.orchestra.run.smithi081:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.6.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 6 2019-11-25T09:15:18.130 INFO:tasks.ceph.osd.6:Started 2019-11-25T09:15:18.131 DEBUG:teuthology.misc:running osd.7 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/osd.7.log', '--time-stamp=yes', '--tool=memcheck'] 2019-11-25T09:15:18.131 INFO:tasks.ceph.osd.7:Restarting daemon 2019-11-25T09:15:18.131 INFO:teuthology.orchestra.run.smithi081:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/osd.7.log --time-stamp=yes --tool=memcheck ceph-osd -f --cluster ceph -i 7 2019-11-25T09:15:18.135 INFO:tasks.ceph.osd.7:Started 2019-11-25T09:15:18.136 INFO:tasks.ceph:Waiting for OSDs to come up 2019-11-25T09:15:18.942 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-11-25T09:15:19.650 INFO:teuthology.misc.health.smithi153.stdout: 2019-11-25T09:15:19.650 INFO:teuthology.misc.health.smithi153.stdout:{"epoch":10,"fsid":"89933446-017a-4383-871e-aaf9415b6b73","created":"2019-11-25T09:15:03.372369+0000","modified":"2019-11-25T09:15:17.840458+0000","last_up_change":"0.000000","last_in_change":"0.000000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"45513261-33b5-44de-a471-c28df1efe738","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"13974267-191f-4964-8d94-59b9df2b785e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"af214156-9298-45bd-a496-38c087d74f40","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fa7627c3-b1e2-4470-896e-c7ae3e921848","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"180d6e76-1a57-40c1-b27b-33b9b2c22f5d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f43c89af-c05e-4f72-959b-e71ac857cf5e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":6,"uuid":"d92af41b-8ff9-4727-91ea-1a4c924fc90e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7e9cedf9-ab32-48f1-8afc-14140c898611","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-25T09:15:19.660 DEBUG:teuthology.misc:0 of 8 OSDs are up 2019-11-25T09:15:21.934 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T09:15:21.923+0000 9b0d800 -1 Falling back to public interface 2019-11-25T09:15:21.956 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T09:15:21.946+0000 9b0d800 -1 Falling back to public interface 2019-11-25T09:15:22.017 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T09:15:22.001+0000 9b0d800 -1 Falling back to public interface 2019-11-25T09:15:22.024 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T09:15:22.010+0000 9b0d800 -1 Falling back to public interface 2019-11-25T09:15:22.827 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T09:15:22.818+0000 9b0d800 -1 Falling back to public interface 2019-11-25T09:15:22.901 INFO:tasks.ceph.osd.7.smithi081.stderr:==00:00:00:04.651 9717== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:22.913 INFO:tasks.ceph.osd.5.smithi081.stderr:==00:00:00:04.662 9718== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:22.970 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T09:15:22.960+0000 9b0d800 -1 Falling back to public interface 2019-11-25T09:15:22.971 INFO:tasks.ceph.osd.1.smithi153.stderr:==00:00:00:04.749 9693== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:22.972 INFO:tasks.ceph.osd.0.smithi153.stderr:==00:00:00:04.746 9694== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:23.108 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T09:15:23.088+0000 9b0d800 -1 Falling back to public interface 2019-11-25T09:15:23.157 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T09:15:23.146+0000 9b0d800 -1 Falling back to public interface 2019-11-25T09:15:23.297 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T09:15:23.276+0000 9b0d800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:15:23.298 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T09:15:23.291+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:23.298 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T09:15:23.294+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:23.357 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T09:15:23.336+0000 9b0d800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:15:23.358 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T09:15:23.338+0000 9b0d800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:15:23.365 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T09:15:23.351+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:23.372 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T09:15:23.354+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:23.373 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T09:15:23.355+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:23.380 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T09:15:23.357+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:23.394 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T09:15:23.347+0000 9b0d800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:15:23.430 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T09:15:23.369+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:23.430 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T09:15:23.373+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:23.723 INFO:tasks.ceph.osd.2.smithi153.stderr:==00:00:00:05.474 9695== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:23.855 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T09:15:23.853+0000 9b0d800 -1 osd.5 0 log_to_monitors {default=true} 2019-11-25T09:15:23.886 INFO:tasks.ceph.osd.3.smithi153.stderr:==00:00:00:05.631 9696== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:23.932 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T09:15:23.930+0000 9b0d800 -1 osd.7 0 log_to_monitors {default=true} 2019-11-25T09:15:23.954 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T09:15:23.952+0000 9b0d800 -1 osd.0 0 log_to_monitors {default=true} 2019-11-25T09:15:23.968 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T09:15:23.967+0000 9b0d800 -1 osd.1 0 log_to_monitors {default=true} 2019-11-25T09:15:24.031 INFO:tasks.ceph.osd.6.smithi081.stderr:==00:00:00:05.804 9710== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:24.133 INFO:tasks.ceph.osd.4.smithi081.stderr:==00:00:00:05.884 9716== Warning: unimplemented fcntl command: 1036 2019-11-25T09:15:24.145 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T09:15:24.130+0000 9b0d800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:15:24.171 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T09:15:24.147+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:24.188 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T09:15:24.151+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:24.312 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T09:15:24.293+0000 9b0d800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:15:24.347 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T09:15:24.310+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:24.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T09:15:24.316+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:24.443 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T09:15:24.411+0000 9b0d800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:15:24.454 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T09:15:24.427+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:24.458 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T09:15:24.430+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:24.534 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T09:15:24.517+0000 9b0d800 -1 journal FileJournal::_open: disabling aio for non-block journal. Use journal_force_aio to force use of aio anyway 2019-11-25T09:15:24.546 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T09:15:24.532+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:24.557 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T09:15:24.535+0000 9b0d800 -1 journal do_read_entry(8192): bad header magic 2019-11-25T09:15:24.698 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T09:15:24.695+0000 9b0d800 -1 osd.2 0 log_to_monitors {default=true} 2019-11-25T09:15:24.837 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T09:15:24.835+0000 9b0d800 -1 osd.3 0 log_to_monitors {default=true} 2019-11-25T09:15:25.013 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T09:15:25.011+0000 9b0d800 -1 osd.6 0 log_to_monitors {default=true} 2019-11-25T09:15:25.146 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T09:15:25.145+0000 9b0d800 -1 osd.4 0 log_to_monitors {default=true} 2019-11-25T09:15:25.499 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T09:15:25.490+0000 29a59700 -1 osd.5 0 waiting for initial osdmap 2019-11-25T09:15:26.469 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-11-25T09:15:26.707 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T09:15:26.705+0000 29a59700 -1 osd.3 0 waiting for initial osdmap 2019-11-25T09:15:26.722 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T09:15:26.720+0000 29a59700 -1 osd.6 0 waiting for initial osdmap 2019-11-25T09:15:26.742 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T09:15:26.732+0000 29a59700 -1 osd.7 0 waiting for initial osdmap 2019-11-25T09:15:26.771 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T09:15:26.768+0000 29a59700 -1 osd.2 0 waiting for initial osdmap 2019-11-25T09:15:26.789 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T09:15:26.782+0000 29659700 -1 osd.0 0 waiting for initial osdmap 2019-11-25T09:15:26.835 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T09:15:26.814+0000 29a59700 -1 osd.1 0 waiting for initial osdmap 2019-11-25T09:15:27.300 INFO:teuthology.misc.health.smithi153.stdout: 2019-11-25T09:15:27.300 INFO:teuthology.misc.health.smithi153.stdout:{"epoch":13,"fsid":"89933446-017a-4383-871e-aaf9415b6b73","created":"2019-11-25T09:15:03.372369+0000","modified":"2019-11-25T09:15:26.198488+0000","last_up_change":"2019-11-25T09:15:26.198488+0000","last_in_change":"2019-11-25T09:15:26.198488+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"45513261-33b5-44de-a471-c28df1efe738","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"13974267-191f-4964-8d94-59b9df2b785e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"af214156-9298-45bd-a496-38c087d74f40","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"fa7627c3-b1e2-4470-896e-c7ae3e921848","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":4,"uuid":"180d6e76-1a57-40c1-b27b-33b9b2c22f5d","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":5,"uuid":"f43c89af-c05e-4f72-959b-e71ac857cf5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":9718},{"type":"v1","addr":"172.21.15.81:6801","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":9718},{"type":"v1","addr":"172.21.15.81:6803","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":9718},{"type":"v1","addr":"172.21.15.81:6807","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":9718},{"type":"v1","addr":"172.21.15.81:6805","nonce":9718}]},"public_addr":"172.21.15.81:6801/9718","cluster_addr":"172.21.15.81:6803/9718","heartbeat_back_addr":"172.21.15.81:6807/9718","heartbeat_front_addr":"172.21.15.81:6805/9718","state":["exists","up"]},{"osd":6,"uuid":"d92af41b-8ff9-4727-91ea-1a4c924fc90e","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":7,"uuid":"7e9cedf9-ab32-48f1-8afc-14140c898611","up":0,"in":0,"weight":0,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-25T09:15:27.318 DEBUG:teuthology.misc:1 of 8 OSDs are up 2019-11-25T09:15:27.636 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T09:15:27.630+0000 29a59700 -1 osd.4 0 waiting for initial osdmap 2019-11-25T09:15:34.125 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2019-11-25T09:15:35.072 INFO:teuthology.misc.health.smithi153.stdout: 2019-11-25T09:15:35.072 INFO:teuthology.misc.health.smithi153.stdout:{"epoch":16,"fsid":"89933446-017a-4383-871e-aaf9415b6b73","created":"2019-11-25T09:15:03.372369+0000","modified":"2019-11-25T09:15:29.681482+0000","last_up_change":"2019-11-25T09:15:28.314329+0000","last_in_change":"2019-11-25T09:15:28.314329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[],"osds":[{"osd":0,"uuid":"45513261-33b5-44de-a471-c28df1efe738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":9694},{"type":"v1","addr":"172.21.15.153:6803","nonce":9694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":9694},{"type":"v1","addr":"172.21.15.153:6807","nonce":9694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":9694},{"type":"v1","addr":"172.21.15.153:6817","nonce":9694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6809","nonce":9694},{"type":"v1","addr":"172.21.15.153:6811","nonce":9694}]},"public_addr":"172.21.15.153:6803/9694","cluster_addr":"172.21.15.153:6807/9694","heartbeat_back_addr":"172.21.15.153:6817/9694","heartbeat_front_addr":"172.21.15.153:6811/9694","state":["exists","up"]},{"osd":1,"uuid":"13974267-191f-4964-8d94-59b9df2b785e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":9693},{"type":"v1","addr":"172.21.15.153:6805","nonce":9693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":9693},{"type":"v1","addr":"172.21.15.153:6810","nonce":9693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6815","nonce":9693},{"type":"v1","addr":"172.21.15.153:6816","nonce":9693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":9693},{"type":"v1","addr":"172.21.15.153:6813","nonce":9693}]},"public_addr":"172.21.15.153:6805/9693","cluster_addr":"172.21.15.153:6810/9693","heartbeat_back_addr":"172.21.15.153:6816/9693","heartbeat_front_addr":"172.21.15.153:6813/9693","state":["exists","up"]},{"osd":2,"uuid":"af214156-9298-45bd-a496-38c087d74f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":9695},{"type":"v1","addr":"172.21.15.153:6819","nonce":9695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":9695},{"type":"v1","addr":"172.21.15.153:6821","nonce":9695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":9695},{"type":"v1","addr":"172.21.15.153:6825","nonce":9695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":9695},{"type":"v1","addr":"172.21.15.153:6823","nonce":9695}]},"public_addr":"172.21.15.153:6819/9695","cluster_addr":"172.21.15.153:6821/9695","heartbeat_back_addr":"172.21.15.153:6825/9695","heartbeat_front_addr":"172.21.15.153:6823/9695","state":["exists","up"]},{"osd":3,"uuid":"fa7627c3-b1e2-4470-896e-c7ae3e921848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":9696},{"type":"v1","addr":"172.21.15.153:6827","nonce":9696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":9696},{"type":"v1","addr":"172.21.15.153:6829","nonce":9696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":9696},{"type":"v1","addr":"172.21.15.153:6833","nonce":9696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":9696},{"type":"v1","addr":"172.21.15.153:6831","nonce":9696}]},"public_addr":"172.21.15.153:6827/9696","cluster_addr":"172.21.15.153:6829/9696","heartbeat_back_addr":"172.21.15.153:6833/9696","heartbeat_front_addr":"172.21.15.153:6831/9696","state":["exists","up"]},{"osd":4,"uuid":"180d6e76-1a57-40c1-b27b-33b9b2c22f5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":9716},{"type":"v1","addr":"172.21.15.81:6825","nonce":9716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":9716},{"type":"v1","addr":"172.21.15.81:6827","nonce":9716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":9716},{"type":"v1","addr":"172.21.15.81:6831","nonce":9716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":9716},{"type":"v1","addr":"172.21.15.81:6829","nonce":9716}]},"public_addr":"172.21.15.81:6825/9716","cluster_addr":"172.21.15.81:6827/9716","heartbeat_back_addr":"172.21.15.81:6831/9716","heartbeat_front_addr":"172.21.15.81:6829/9716","state":["exists","up"]},{"osd":5,"uuid":"f43c89af-c05e-4f72-959b-e71ac857cf5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":9718},{"type":"v1","addr":"172.21.15.81:6801","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":9718},{"type":"v1","addr":"172.21.15.81:6803","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":9718},{"type":"v1","addr":"172.21.15.81:6807","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":9718},{"type":"v1","addr":"172.21.15.81:6805","nonce":9718}]},"public_addr":"172.21.15.81:6801/9718","cluster_addr":"172.21.15.81:6803/9718","heartbeat_back_addr":"172.21.15.81:6807/9718","heartbeat_front_addr":"172.21.15.81:6805/9718","state":["exists","up"]},{"osd":6,"uuid":"d92af41b-8ff9-4727-91ea-1a4c924fc90e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":9710},{"type":"v1","addr":"172.21.15.81:6817","nonce":9710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":9710},{"type":"v1","addr":"172.21.15.81:6819","nonce":9710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":9710},{"type":"v1","addr":"172.21.15.81:6823","nonce":9710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":9710},{"type":"v1","addr":"172.21.15.81:6821","nonce":9710}]},"public_addr":"172.21.15.81:6817/9710","cluster_addr":"172.21.15.81:6819/9710","heartbeat_back_addr":"172.21.15.81:6823/9710","heartbeat_front_addr":"172.21.15.81:6821/9710","state":["exists","up"]},{"osd":7,"uuid":"7e9cedf9-ab32-48f1-8afc-14140c898611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":9717},{"type":"v1","addr":"172.21.15.81:6809","nonce":9717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":9717},{"type":"v1","addr":"172.21.15.81:6811","nonce":9717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":9717},{"type":"v1","addr":"172.21.15.81:6815","nonce":9717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":9717},{"type":"v1","addr":"172.21.15.81:6813","nonce":9717}]},"public_addr":"172.21.15.81:6809/9717","cluster_addr":"172.21.15.81:6811/9717","heartbeat_back_addr":"172.21.15.81:6815/9717","heartbeat_front_addr":"172.21.15.81:6813/9717","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.048117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.067518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.868737+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.934455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:26.200930+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:24.940504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:26.131736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:24.981215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-25T09:15:35.090 DEBUG:teuthology.misc:8 of 8 OSDs are up 2019-11-25T09:15:35.090 INFO:tasks.ceph:Creating RBD pool 2019-11-25T09:15:35.090 INFO:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd pool create rbd 8 2019-11-25T09:15:37.399 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'rbd' created 2019-11-25T09:15:37.417 INFO:teuthology.orchestra.run.smithi153:> sudo ceph --cluster ceph osd pool application enable rbd rbd --yes-i-really-mean-it 2019-11-25T09:15:38.737 INFO:teuthology.orchestra.run.smithi153.stderr:enabled application 'rbd' on pool 'rbd' 2019-11-25T09:15:38.750 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2019-11-25T09:15:38.750 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-11-25T09:15:38.751 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2019-11-25T09:15:39.494 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:15:39.494 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":18,"fsid":"89933446-017a-4383-871e-aaf9415b6b73","created":"2019-11-25T09:15:03.372369+0000","modified":"2019-11-25T09:15:38.472025+0000","last_up_change":"2019-11-25T09:15:28.314329+0000","last_in_change":"2019-11-25T09:15:28.314329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-11-25T09:15:36.088689+0000","flags":32769,"flags_names":"hashpspool,creating","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"45513261-33b5-44de-a471-c28df1efe738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":9694},{"type":"v1","addr":"172.21.15.153:6803","nonce":9694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":9694},{"type":"v1","addr":"172.21.15.153:6807","nonce":9694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":9694},{"type":"v1","addr":"172.21.15.153:6817","nonce":9694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6809","nonce":9694},{"type":"v1","addr":"172.21.15.153:6811","nonce":9694}]},"public_addr":"172.21.15.153:6803/9694","cluster_addr":"172.21.15.153:6807/9694","heartbeat_back_addr":"172.21.15.153:6817/9694","heartbeat_front_addr":"172.21.15.153:6811/9694","state":["exists","up"]},{"osd":1,"uuid":"13974267-191f-4964-8d94-59b9df2b785e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":9693},{"type":"v1","addr":"172.21.15.153:6805","nonce":9693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":9693},{"type":"v1","addr":"172.21.15.153:6810","nonce":9693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6815","nonce":9693},{"type":"v1","addr":"172.21.15.153:6816","nonce":9693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":9693},{"type":"v1","addr":"172.21.15.153:6813","nonce":9693}]},"public_addr":"172.21.15.153:6805/9693","cluster_addr":"172.21.15.153:6810/9693","heartbeat_back_addr":"172.21.15.153:6816/9693","heartbeat_front_addr":"172.21.15.153:6813/9693","state":["exists","up"]},{"osd":2,"uuid":"af214156-9298-45bd-a496-38c087d74f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":9695},{"type":"v1","addr":"172.21.15.153:6819","nonce":9695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":9695},{"type":"v1","addr":"172.21.15.153:6821","nonce":9695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":9695},{"type":"v1","addr":"172.21.15.153:6825","nonce":9695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":9695},{"type":"v1","addr":"172.21.15.153:6823","nonce":9695}]},"public_addr":"172.21.15.153:6819/9695","cluster_addr":"172.21.15.153:6821/9695","heartbeat_back_addr":"172.21.15.153:6825/9695","heartbeat_front_addr":"172.21.15.153:6823/9695","state":["exists","up"]},{"osd":3,"uuid":"fa7627c3-b1e2-4470-896e-c7ae3e921848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":9696},{"type":"v1","addr":"172.21.15.153:6827","nonce":9696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":9696},{"type":"v1","addr":"172.21.15.153:6829","nonce":9696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":9696},{"type":"v1","addr":"172.21.15.153:6833","nonce":9696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":9696},{"type":"v1","addr":"172.21.15.153:6831","nonce":9696}]},"public_addr":"172.21.15.153:6827/9696","cluster_addr":"172.21.15.153:6829/9696","heartbeat_back_addr":"172.21.15.153:6833/9696","heartbeat_front_addr":"172.21.15.153:6831/9696","state":["exists","up"]},{"osd":4,"uuid":"180d6e76-1a57-40c1-b27b-33b9b2c22f5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":9716},{"type":"v1","addr":"172.21.15.81:6825","nonce":9716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":9716},{"type":"v1","addr":"172.21.15.81:6827","nonce":9716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":9716},{"type":"v1","addr":"172.21.15.81:6831","nonce":9716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":9716},{"type":"v1","addr":"172.21.15.81:6829","nonce":9716}]},"public_addr":"172.21.15.81:6825/9716","cluster_addr":"172.21.15.81:6827/9716","heartbeat_back_addr":"172.21.15.81:6831/9716","heartbeat_front_addr":"172.21.15.81:6829/9716","state":["exists","up"]},{"osd":5,"uuid":"f43c89af-c05e-4f72-959b-e71ac857cf5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":9718},{"type":"v1","addr":"172.21.15.81:6801","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":9718},{"type":"v1","addr":"172.21.15.81:6803","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":9718},{"type":"v1","addr":"172.21.15.81:6807","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":9718},{"type":"v1","addr":"172.21.15.81:6805","nonce":9718}]},"public_addr":"172.21.15.81:6801/9718","cluster_addr":"172.21.15.81:6803/9718","heartbeat_back_addr":"172.21.15.81:6807/9718","heartbeat_front_addr":"172.21.15.81:6805/9718","state":["exists","up"]},{"osd":6,"uuid":"d92af41b-8ff9-4727-91ea-1a4c924fc90e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":9710},{"type":"v1","addr":"172.21.15.81:6817","nonce":9710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":9710},{"type":"v1","addr":"172.21.15.81:6819","nonce":9710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":9710},{"type":"v1","addr":"172.21.15.81:6823","nonce":9710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":9710},{"type":"v1","addr":"172.21.15.81:6821","nonce":9710}]},"public_addr":"172.21.15.81:6817/9710","cluster_addr":"172.21.15.81:6819/9710","heartbeat_back_addr":"172.21.15.81:6823/9710","heartbeat_front_addr":"172.21.15.81:6821/9710","state":["exists","up"]},{"osd":7,"uuid":"7e9cedf9-ab32-48f1-8afc-14140c898611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":9717},{"type":"v1","addr":"172.21.15.81:6809","nonce":9717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":9717},{"type":"v1","addr":"172.21.15.81:6811","nonce":9717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":9717},{"type":"v1","addr":"172.21.15.81:6815","nonce":9717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":9717},{"type":"v1","addr":"172.21.15.81:6813","nonce":9717}]},"public_addr":"172.21.15.81:6809/9717","cluster_addr":"172.21.15.81:6811/9717","heartbeat_back_addr":"172.21.15.81:6815/9717","heartbeat_front_addr":"172.21.15.81:6813/9717","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.048117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.067518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.868737+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.934455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:26.200930+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:24.940504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:26.131736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:24.981215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-25T09:15:39.515 INFO:tasks.ceph.ceph_manager.ceph:[{u'cache_target_full_ratio_micro': 800000, u'fast_read': False, u'stripe_width': 0, u'flags_names': u'hashpspool,creating', u'tier_of': -1, u'hit_set_grade_decay_rate': 0, u'pg_placement_num': 8, u'use_gmt_hitset': True, u'cache_target_dirty_ratio_micro': 400000, u'create_time': u'2019-11-25T09:15:36.088689+0000', u'snap_epoch': 0, u'erasure_code_profile': u'', u'pg_autoscale_mode': u'off', u'snap_seq': 0, u'expected_num_objects': 0, u'size': 2, u'pg_num_pending': 8, u'auid': 0, u'cache_min_flush_age': 0, u'hit_set_period': 0, u'min_read_recency_for_promote': 0, u'target_max_objects': 0, u'pg_placement_num_target': 8, u'pg_num': 8, u'type': 1, u'grade_table': [], u'pool_name': u'rbd', u'cache_min_evict_age': 0, u'snap_mode': u'selfmanaged', u'pg_num_target': 8, u'cache_mode': u'none', u'min_size': 1, u'cache_target_dirty_high_ratio_micro': 600000, u'object_hash': 2, u'last_pg_merge_meta': {u'ready_epoch': 0, u'source_version': u"0'0", u'source_pgid': u'0.0', u'last_epoch_clean': 0, u'target_version': u"0'0", u'last_epoch_started': 0}, u'write_tier': -1, u'last_force_op_resend_preluminous': u'0', u'pool': 1, u'removed_snaps': u'[]', u'crush_rule': 0, u'tiers': [], u'hit_set_params': {u'type': u'none'}, u'last_force_op_resend': u'0', u'pool_snaps': [], u'quota_max_objects': 0, u'last_force_op_resend_prenautilus': u'0', u'application_metadata': {u'rbd': {}}, u'options': {}, u'hit_set_count': 0, u'flags': 32769, u'target_max_bytes': 0, u'quota_max_bytes': 0, u'hit_set_search_last_n': 0, u'last_change': u'18', u'min_write_recency_for_promote': 0, u'read_tier': -1}] 2019-11-25T09:15:39.516 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool get rbd pg_num 2019-11-25T09:15:40.451 INFO:teuthology.orchestra.run.smithi153.stdout:pg_num: 8 2019-11-25T09:15:40.469 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2019-11-25T09:15:40.469 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2019-11-25T09:15:40.469 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2019-11-25T09:15:41.349 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:15:41.369 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":3,"active_gid":4100,"active_name":"y","active_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6800","nonce":8740},{"type":"v1","addr":"172.21.15.153:6801","nonce":8740}]},"active_addr":"172.21.15.153:6801/8740","active_change":"2019-11-25T09:15:07.170431+0000","active_mgr_features":4540138292836696063,"available":true,"standbys":[{"gid":4108,"name":"x","mgr_features":4540138292836696063,"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"float","level":"advanced","flags":1,"default_value":"0.01","min":"0","max":"1","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the ratio between the fullest and least-full OSD is below this value then we stop trying to optimize placement.","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}]}],"modules":["iostat","restful"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"str","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]}}},{"name":"ansible","can_run":true,"error_string":"","module_options":{"ca_bundle":{"name":"ca_bundle","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_location":{"name":"server_location","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_server":{"name":"verify_server","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2400","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"7","min":"0","max":"7","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 or 7 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"none","min":"","max":"","enum_allowed":["crush-compat","none","upmap"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"float","level":"advanced","flags":1,"default_value":"0.01","min":"0","max":"1","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the ratio between the fullest and least-full OSD is below this value then we stop trying to optimize placement.","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_cephfs":{"name":"FEATURE_TOGGLE_cephfs","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_iscsi":{"name":"FEATURE_TOGGLE_iscsi","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_mirroring":{"name":"FEATURE_TOGGLE_mirroring","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rbd":{"name":"FEATURE_TOGGLE_rbd","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_rgw":{"name":"FEATURE_TOGGLE_rgw","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"str","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"str","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_HOST":{"name":"RGW_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_PORT":{"name":"RGW_API_PORT","type":"str","level":"advanced","flags":0,"default_value":"80","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SCHEME":{"name":"RGW_API_SCHEME","type":"str","level":"advanced","flags":0,"default_value":"http","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"str","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_USER_ID":{"name":"RGW_API_USER_ID","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"deepsea","can_run":true,"error_string":"","module_options":{"salt_api_eauth":{"name":"salt_api_eauth","type":"str","level":"advanced","flags":0,"default_value":"sharedsecret","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_password":{"name":"salt_api_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_url":{"name":"salt_api_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"salt_api_username":{"name":"salt_api_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_cloud","can_run":true,"error_string":"","module_options":{"diskprediction_cert_context":{"name":"diskprediction_cert_context","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_default_authority":{"name":"diskprediction_default_authority","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_password":{"name":"diskprediction_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_port":{"name":"diskprediction_port","type":"str","level":"advanced","flags":0,"default_value":"31400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_retrieve_prediction_interval":{"name":"diskprediction_retrieve_prediction_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_server":{"name":"diskprediction_server","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_ssl_target_name_override":{"name":"diskprediction_ssl_target_name_override","type":"str","level":"advanced","flags":0,"default_value":"localhost","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_metrics_interval":{"name":"diskprediction_upload_metrics_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_upload_smart_interval":{"name":"diskprediction_upload_smart_interval","type":"str","level":"advanced","flags":0,"default_value":"43200","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"diskprediction_user":{"name":"diskprediction_user","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"str","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"str","level":"advanced","flags":0,"default_value":"30","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"str","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"str","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{}},{"name":"iostat","can_run":true,"error_string":"","module_options":{}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator_cli","can_run":true,"error_string":"","module_options":{"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"persist_interval":{"name":"persist_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how frequently to persist completed events","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{}},{"name":"restful","can_run":true,"error_string":"","module_options":{"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"ssh","can_run":true,"error_string":"","module_options":{"inventory_cache_timeout":{"name":"inventory_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"seconds to cache device inventory","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["ceph-daemon-package","root"],"desc":"mode for remote execution of ceph-daemon","long_desc":"","tags":[],"see_also":[]},"service_cache_timeout":{"name":"service_cache_timeout","type":"str","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"seconds to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"customized SSH config file to connect to managed hosts","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{}},{"name":"volumes","can_run":true,"error_string":"","module_options":{}},{"name":"zabbix","can_run":true,"error_string":"","module_options":{"discovery_interval":{"name":"discovery_interval","type":"str","level":"advanced","flags":0,"default_value":"100","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"identifier":{"name":"identifier","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_host":{"name":"zabbix_host","type":"str","level":"advanced","flags":0,"default_value":"None","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_port":{"name":"zabbix_port","type":"int","level":"advanced","flags":0,"default_value":"10051","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"zabbix_sender":{"name":"zabbix_sender","type":"str","level":"advanced","flags":0,"default_value":"/usr/bin/zabbix_sender","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"nautilus":["balancer","crash","devicehealth","orchestrator_cli","progress","rbd_support","status","volumes"],"octopus":["balancer","crash","devicehealth","orchestrator_cli","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]}} 2019-11-25T09:15:41.373 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2019-11-25T09:15:41.373 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2019-11-25T09:15:41.373 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-11-25T09:15:42.364 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:15:42.364 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":20,"fsid":"89933446-017a-4383-871e-aaf9415b6b73","created":"2019-11-25T09:15:03.372369+0000","modified":"2019-11-25T09:15:40.817638+0000","last_up_change":"2019-11-25T09:15:28.314329+0000","last_in_change":"2019-11-25T09:15:28.314329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-11-25T09:15:36.088689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"45513261-33b5-44de-a471-c28df1efe738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":9694},{"type":"v1","addr":"172.21.15.153:6803","nonce":9694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":9694},{"type":"v1","addr":"172.21.15.153:6807","nonce":9694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":9694},{"type":"v1","addr":"172.21.15.153:6817","nonce":9694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6809","nonce":9694},{"type":"v1","addr":"172.21.15.153:6811","nonce":9694}]},"public_addr":"172.21.15.153:6803/9694","cluster_addr":"172.21.15.153:6807/9694","heartbeat_back_addr":"172.21.15.153:6817/9694","heartbeat_front_addr":"172.21.15.153:6811/9694","state":["exists","up"]},{"osd":1,"uuid":"13974267-191f-4964-8d94-59b9df2b785e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":9693},{"type":"v1","addr":"172.21.15.153:6805","nonce":9693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":9693},{"type":"v1","addr":"172.21.15.153:6810","nonce":9693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6815","nonce":9693},{"type":"v1","addr":"172.21.15.153:6816","nonce":9693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":9693},{"type":"v1","addr":"172.21.15.153:6813","nonce":9693}]},"public_addr":"172.21.15.153:6805/9693","cluster_addr":"172.21.15.153:6810/9693","heartbeat_back_addr":"172.21.15.153:6816/9693","heartbeat_front_addr":"172.21.15.153:6813/9693","state":["exists","up"]},{"osd":2,"uuid":"af214156-9298-45bd-a496-38c087d74f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":9695},{"type":"v1","addr":"172.21.15.153:6819","nonce":9695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":9695},{"type":"v1","addr":"172.21.15.153:6821","nonce":9695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":9695},{"type":"v1","addr":"172.21.15.153:6825","nonce":9695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":9695},{"type":"v1","addr":"172.21.15.153:6823","nonce":9695}]},"public_addr":"172.21.15.153:6819/9695","cluster_addr":"172.21.15.153:6821/9695","heartbeat_back_addr":"172.21.15.153:6825/9695","heartbeat_front_addr":"172.21.15.153:6823/9695","state":["exists","up"]},{"osd":3,"uuid":"fa7627c3-b1e2-4470-896e-c7ae3e921848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":9696},{"type":"v1","addr":"172.21.15.153:6827","nonce":9696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":9696},{"type":"v1","addr":"172.21.15.153:6829","nonce":9696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":9696},{"type":"v1","addr":"172.21.15.153:6833","nonce":9696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":9696},{"type":"v1","addr":"172.21.15.153:6831","nonce":9696}]},"public_addr":"172.21.15.153:6827/9696","cluster_addr":"172.21.15.153:6829/9696","heartbeat_back_addr":"172.21.15.153:6833/9696","heartbeat_front_addr":"172.21.15.153:6831/9696","state":["exists","up"]},{"osd":4,"uuid":"180d6e76-1a57-40c1-b27b-33b9b2c22f5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":9716},{"type":"v1","addr":"172.21.15.81:6825","nonce":9716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":9716},{"type":"v1","addr":"172.21.15.81:6827","nonce":9716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":9716},{"type":"v1","addr":"172.21.15.81:6831","nonce":9716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":9716},{"type":"v1","addr":"172.21.15.81:6829","nonce":9716}]},"public_addr":"172.21.15.81:6825/9716","cluster_addr":"172.21.15.81:6827/9716","heartbeat_back_addr":"172.21.15.81:6831/9716","heartbeat_front_addr":"172.21.15.81:6829/9716","state":["exists","up"]},{"osd":5,"uuid":"f43c89af-c05e-4f72-959b-e71ac857cf5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":9718},{"type":"v1","addr":"172.21.15.81:6801","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":9718},{"type":"v1","addr":"172.21.15.81:6803","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":9718},{"type":"v1","addr":"172.21.15.81:6807","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":9718},{"type":"v1","addr":"172.21.15.81:6805","nonce":9718}]},"public_addr":"172.21.15.81:6801/9718","cluster_addr":"172.21.15.81:6803/9718","heartbeat_back_addr":"172.21.15.81:6807/9718","heartbeat_front_addr":"172.21.15.81:6805/9718","state":["exists","up"]},{"osd":6,"uuid":"d92af41b-8ff9-4727-91ea-1a4c924fc90e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":9710},{"type":"v1","addr":"172.21.15.81:6817","nonce":9710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":9710},{"type":"v1","addr":"172.21.15.81:6819","nonce":9710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":9710},{"type":"v1","addr":"172.21.15.81:6823","nonce":9710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":9710},{"type":"v1","addr":"172.21.15.81:6821","nonce":9710}]},"public_addr":"172.21.15.81:6817/9710","cluster_addr":"172.21.15.81:6819/9710","heartbeat_back_addr":"172.21.15.81:6823/9710","heartbeat_front_addr":"172.21.15.81:6821/9710","state":["exists","up"]},{"osd":7,"uuid":"7e9cedf9-ab32-48f1-8afc-14140c898611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":9717},{"type":"v1","addr":"172.21.15.81:6809","nonce":9717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":9717},{"type":"v1","addr":"172.21.15.81:6811","nonce":9717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":9717},{"type":"v1","addr":"172.21.15.81:6815","nonce":9717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":9717},{"type":"v1","addr":"172.21.15.81:6813","nonce":9717}]},"public_addr":"172.21.15.81:6809/9717","cluster_addr":"172.21.15.81:6811/9717","heartbeat_back_addr":"172.21.15.81:6815/9717","heartbeat_front_addr":"172.21.15.81:6813/9717","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.048117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.067518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.868737+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.934455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:26.200930+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:24.940504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:26.131736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:24.981215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-25T09:15:42.392 INFO:tasks.ceph.ceph_manager.ceph:all up! 2019-11-25T09:15:42.392 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2019-11-25T09:15:43.389 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:15:43.389 INFO:teuthology.orchestra.run.smithi153.stdout:{"epoch":20,"fsid":"89933446-017a-4383-871e-aaf9415b6b73","created":"2019-11-25T09:15:03.372369+0000","modified":"2019-11-25T09:15:40.817638+0000","last_up_change":"2019-11-25T09:15:28.314329+0000","last_in_change":"2019-11-25T09:15:28.314329+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":8,"require_min_compat_client":"jewel","min_compat_client":"jewel","require_osd_release":"octopus","pools":[{"pool":1,"pool_name":"rbd","create_time":"2019-11-25T09:15:36.088689+0000","flags":1,"flags_names":"hashpspool","type":1,"size":2,"min_size":1,"crush_rule":0,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"20","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"options":{},"application_metadata":{"rbd":{}}}],"osds":[{"osd":0,"uuid":"45513261-33b5-44de-a471-c28df1efe738","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6802","nonce":9694},{"type":"v1","addr":"172.21.15.153:6803","nonce":9694}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6806","nonce":9694},{"type":"v1","addr":"172.21.15.153:6807","nonce":9694}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6814","nonce":9694},{"type":"v1","addr":"172.21.15.153:6817","nonce":9694}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6809","nonce":9694},{"type":"v1","addr":"172.21.15.153:6811","nonce":9694}]},"public_addr":"172.21.15.153:6803/9694","cluster_addr":"172.21.15.153:6807/9694","heartbeat_back_addr":"172.21.15.153:6817/9694","heartbeat_front_addr":"172.21.15.153:6811/9694","state":["exists","up"]},{"osd":1,"uuid":"13974267-191f-4964-8d94-59b9df2b785e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6804","nonce":9693},{"type":"v1","addr":"172.21.15.153:6805","nonce":9693}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6808","nonce":9693},{"type":"v1","addr":"172.21.15.153:6810","nonce":9693}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6815","nonce":9693},{"type":"v1","addr":"172.21.15.153:6816","nonce":9693}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6812","nonce":9693},{"type":"v1","addr":"172.21.15.153:6813","nonce":9693}]},"public_addr":"172.21.15.153:6805/9693","cluster_addr":"172.21.15.153:6810/9693","heartbeat_back_addr":"172.21.15.153:6816/9693","heartbeat_front_addr":"172.21.15.153:6813/9693","state":["exists","up"]},{"osd":2,"uuid":"af214156-9298-45bd-a496-38c087d74f40","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6818","nonce":9695},{"type":"v1","addr":"172.21.15.153:6819","nonce":9695}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6820","nonce":9695},{"type":"v1","addr":"172.21.15.153:6821","nonce":9695}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6824","nonce":9695},{"type":"v1","addr":"172.21.15.153:6825","nonce":9695}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6822","nonce":9695},{"type":"v1","addr":"172.21.15.153:6823","nonce":9695}]},"public_addr":"172.21.15.153:6819/9695","cluster_addr":"172.21.15.153:6821/9695","heartbeat_back_addr":"172.21.15.153:6825/9695","heartbeat_front_addr":"172.21.15.153:6823/9695","state":["exists","up"]},{"osd":3,"uuid":"fa7627c3-b1e2-4470-896e-c7ae3e921848","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":14,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6826","nonce":9696},{"type":"v1","addr":"172.21.15.153:6827","nonce":9696}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6828","nonce":9696},{"type":"v1","addr":"172.21.15.153:6829","nonce":9696}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6832","nonce":9696},{"type":"v1","addr":"172.21.15.153:6833","nonce":9696}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.153:6830","nonce":9696},{"type":"v1","addr":"172.21.15.153:6831","nonce":9696}]},"public_addr":"172.21.15.153:6827/9696","cluster_addr":"172.21.15.153:6829/9696","heartbeat_back_addr":"172.21.15.153:6833/9696","heartbeat_front_addr":"172.21.15.153:6831/9696","state":["exists","up"]},{"osd":4,"uuid":"180d6e76-1a57-40c1-b27b-33b9b2c22f5d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6824","nonce":9716},{"type":"v1","addr":"172.21.15.81:6825","nonce":9716}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6826","nonce":9716},{"type":"v1","addr":"172.21.15.81:6827","nonce":9716}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6830","nonce":9716},{"type":"v1","addr":"172.21.15.81:6831","nonce":9716}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6828","nonce":9716},{"type":"v1","addr":"172.21.15.81:6829","nonce":9716}]},"public_addr":"172.21.15.81:6825/9716","cluster_addr":"172.21.15.81:6827/9716","heartbeat_back_addr":"172.21.15.81:6831/9716","heartbeat_front_addr":"172.21.15.81:6829/9716","state":["exists","up"]},{"osd":5,"uuid":"f43c89af-c05e-4f72-959b-e71ac857cf5e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":13,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6800","nonce":9718},{"type":"v1","addr":"172.21.15.81:6801","nonce":9718}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6802","nonce":9718},{"type":"v1","addr":"172.21.15.81:6803","nonce":9718}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6806","nonce":9718},{"type":"v1","addr":"172.21.15.81:6807","nonce":9718}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6804","nonce":9718},{"type":"v1","addr":"172.21.15.81:6805","nonce":9718}]},"public_addr":"172.21.15.81:6801/9718","cluster_addr":"172.21.15.81:6803/9718","heartbeat_back_addr":"172.21.15.81:6807/9718","heartbeat_front_addr":"172.21.15.81:6805/9718","state":["exists","up"]},{"osd":6,"uuid":"d92af41b-8ff9-4727-91ea-1a4c924fc90e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6816","nonce":9710},{"type":"v1","addr":"172.21.15.81:6817","nonce":9710}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6818","nonce":9710},{"type":"v1","addr":"172.21.15.81:6819","nonce":9710}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6822","nonce":9710},{"type":"v1","addr":"172.21.15.81:6823","nonce":9710}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6820","nonce":9710},{"type":"v1","addr":"172.21.15.81:6821","nonce":9710}]},"public_addr":"172.21.15.81:6817/9710","cluster_addr":"172.21.15.81:6819/9710","heartbeat_back_addr":"172.21.15.81:6823/9710","heartbeat_front_addr":"172.21.15.81:6821/9710","state":["exists","up"]},{"osd":7,"uuid":"7e9cedf9-ab32-48f1-8afc-14140c898611","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":15,"up_thru":17,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6808","nonce":9717},{"type":"v1","addr":"172.21.15.81:6809","nonce":9717}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6810","nonce":9717},{"type":"v1","addr":"172.21.15.81:6811","nonce":9717}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6814","nonce":9717},{"type":"v1","addr":"172.21.15.81:6815","nonce":9717}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"172.21.15.81:6812","nonce":9717},{"type":"v1","addr":"172.21.15.81:6813","nonce":9717}]},"public_addr":"172.21.15.81:6809/9717","cluster_addr":"172.21.15.81:6811/9717","heartbeat_back_addr":"172.21.15.81:6815/9717","heartbeat_front_addr":"172.21.15.81:6813/9717","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.048117+0000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.067518+0000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.868737+0000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:25.934455+0000","dead_epoch":0},{"osd":4,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:26.200930+0000","dead_epoch":0},{"osd":5,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:24.940504+0000","dead_epoch":0},{"osd":6,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:26.131736+0000","dead_epoch":0},{"osd":7,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4540138292836696063,"old_weight":0,"last_purged_snaps_scrub":"2019-11-25T09:15:24.981215+0000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_temp":[],"primary_temp":[],"blacklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"jerasure","ruleset-failure-domain":"osd","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{}} 2019-11-25T09:15:43.405 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2019-11-25T09:15:43.926 INFO:teuthology.orchestra.run.smithi153.stdout:64424509447 2019-11-25T09:15:43.926 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2019-11-25T09:15:44.495 INFO:teuthology.orchestra.run.smithi153.stdout:64424509447 2019-11-25T09:15:44.496 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2019-11-25T09:15:44.979 INFO:teuthology.orchestra.run.smithi153.stdout:64424509447 2019-11-25T09:15:44.980 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2019-11-25T09:15:45.588 INFO:teuthology.orchestra.run.smithi153.stdout:60129542150 2019-11-25T09:15:45.589 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.4 flush_pg_stats 2019-11-25T09:15:46.220 INFO:teuthology.orchestra.run.smithi153.stdout:64424509446 2019-11-25T09:15:46.221 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.5 flush_pg_stats 2019-11-25T09:15:46.898 INFO:teuthology.orchestra.run.smithi153.stdout:55834574855 2019-11-25T09:15:46.898 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.6 flush_pg_stats 2019-11-25T09:15:47.495 INFO:teuthology.orchestra.run.smithi153.stdout:64424509447 2019-11-25T09:15:47.496 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.7 flush_pg_stats 2019-11-25T09:15:48.164 INFO:teuthology.orchestra.run.smithi153.stdout:64424509447 2019-11-25T09:15:48.164 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2019-11-25T09:15:49.030 INFO:teuthology.orchestra.run.smithi153.stdout:64424509447 2019-11-25T09:15:49.043 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.0 2019-11-25T09:15:49.043 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2019-11-25T09:15:49.869 INFO:teuthology.orchestra.run.smithi153.stdout:64424509447 2019-11-25T09:15:49.887 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509447 for osd.1 2019-11-25T09:15:49.887 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2019-11-25T09:15:50.674 INFO:teuthology.orchestra.run.smithi153.stdout:64424509448 2019-11-25T09:15:50.687 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.2 2019-11-25T09:15:50.687 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2019-11-25T09:15:51.537 INFO:teuthology.orchestra.run.smithi153.stdout:60129542151 2019-11-25T09:15:51.553 INFO:tasks.ceph.ceph_manager.ceph:need seq 60129542150 got 60129542151 for osd.3 2019-11-25T09:15:51.554 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.4 2019-11-25T09:15:52.302 INFO:teuthology.orchestra.run.smithi153.stdout:64424509447 2019-11-25T09:15:52.321 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509446 got 64424509447 for osd.4 2019-11-25T09:15:52.321 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.5 2019-11-25T09:15:53.166 INFO:teuthology.orchestra.run.smithi153.stdout:55834574856 2019-11-25T09:15:53.181 INFO:tasks.ceph.ceph_manager.ceph:need seq 55834574855 got 55834574856 for osd.5 2019-11-25T09:15:53.181 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.6 2019-11-25T09:15:54.049 INFO:teuthology.orchestra.run.smithi153.stdout:64424509448 2019-11-25T09:15:54.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.6 2019-11-25T09:15:54.065 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.7 2019-11-25T09:15:54.872 INFO:teuthology.orchestra.run.smithi153.stdout:64424509448 2019-11-25T09:15:54.884 INFO:tasks.ceph.ceph_manager.ceph:need seq 64424509447 got 64424509448 for osd.7 2019-11-25T09:15:54.885 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2019-11-25T09:15:54.885 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-11-25T09:15:55.601 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2019-11-25T09:15:55.602 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:15:55.617 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2019-11-25T09:15:55.372706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":1699648,"kb_used_data":747895392,"kb_used_omap":306,"kb_used_meta":0,"kb_avail":747895392,"statfs":{"total":767585320960,"available":765844881408,"internally_reserved":0,"allocated":765844881408,"data_stored":765844881408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":314204,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.258289+0000","last_change":"2019-11-25T09:15:39.326784+0000","last_active":"2019-11-25T09:15:40.258289+0000","last_peered":"2019-11-25T09:15:40.258289+0000","last_clean":"2019-11-25T09:15:40.258289+0000","last_became_active":"2019-11-25T09:15:39.293677+0000","last_became_peered":"2019-11-25T09:15:39.293677+0000","last_unstale":"2019-11-25T09:15:40.258289+0000","last_undegraded":"2019-11-25T09:15:40.258289+0000","last_fullsized":"2019-11-25T09:15:40.258289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.497509+0000","last_change":"2019-11-25T09:15:40.475676+0000","last_active":"2019-11-25T09:15:40.497509+0000","last_peered":"2019-11-25T09:15:40.497509+0000","last_clean":"2019-11-25T09:15:40.497509+0000","last_became_active":"2019-11-25T09:15:40.409422+0000","last_became_peered":"2019-11-25T09:15:40.409422+0000","last_unstale":"2019-11-25T09:15:40.497509+0000","last_undegraded":"2019-11-25T09:15:40.497509+0000","last_fullsized":"2019-11-25T09:15:40.497509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.263681+0000","last_change":"2019-11-25T09:15:39.364197+0000","last_active":"2019-11-25T09:15:40.263681+0000","last_peered":"2019-11-25T09:15:40.263681+0000","last_clean":"2019-11-25T09:15:40.263681+0000","last_became_active":"2019-11-25T09:15:39.310975+0000","last_became_peered":"2019-11-25T09:15:39.310975+0000","last_unstale":"2019-11-25T09:15:40.263681+0000","last_undegraded":"2019-11-25T09:15:40.263681+0000","last_fullsized":"2019-11-25T09:15:40.263681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.266381+0000","last_change":"2019-11-25T09:15:39.082760+0000","last_active":"2019-11-25T09:15:40.266381+0000","last_peered":"2019-11-25T09:15:40.266381+0000","last_clean":"2019-11-25T09:15:40.266381+0000","last_became_active":"2019-11-25T09:15:39.032784+0000","last_became_peered":"2019-11-25T09:15:39.032784+0000","last_unstale":"2019-11-25T09:15:40.266381+0000","last_undegraded":"2019-11-25T09:15:40.266381+0000","last_fullsized":"2019-11-25T09:15:40.266381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.250971+0000","last_change":"2019-11-25T09:15:39.388305+0000","last_active":"2019-11-25T09:15:40.250971+0000","last_peered":"2019-11-25T09:15:40.250971+0000","last_clean":"2019-11-25T09:15:40.250971+0000","last_became_active":"2019-11-25T09:15:39.375469+0000","last_became_peered":"2019-11-25T09:15:39.375469+0000","last_unstale":"2019-11-25T09:15:40.250971+0000","last_undegraded":"2019-11-25T09:15:40.250971+0000","last_fullsized":"2019-11-25T09:15:40.250971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.492415+0000","last_change":"2019-11-25T09:15:40.462512+0000","last_active":"2019-11-25T09:15:40.492415+0000","last_peered":"2019-11-25T09:15:40.492415+0000","last_clean":"2019-11-25T09:15:40.492415+0000","last_became_active":"2019-11-25T09:15:40.381235+0000","last_became_peered":"2019-11-25T09:15:40.381235+0000","last_unstale":"2019-11-25T09:15:40.492415+0000","last_undegraded":"2019-11-25T09:15:40.492415+0000","last_fullsized":"2019-11-25T09:15:40.492415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:39.913253+0000","last_change":"2019-11-25T09:15:39.113495+0000","last_active":"2019-11-25T09:15:39.913253+0000","last_peered":"2019-11-25T09:15:39.913253+0000","last_clean":"2019-11-25T09:15:39.913253+0000","last_became_active":"2019-11-25T09:15:39.089643+0000","last_became_peered":"2019-11-25T09:15:39.089643+0000","last_unstale":"2019-11-25T09:15:39.913253+0000","last_undegraded":"2019-11-25T09:15:39.913253+0000","last_fullsized":"2019-11-25T09:15:39.913253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:39.873005+0000","last_change":"2019-11-25T09:15:39.134492+0000","last_active":"2019-11-25T09:15:39.873005+0000","last_peered":"2019-11-25T09:15:39.873005+0000","last_clean":"2019-11-25T09:15:39.873005+0000","last_became_active":"2019-11-25T09:15:39.091359+0000","last_became_peered":"2019-11-25T09:15:39.091359+0000","last_unstale":"2019-11-25T09:15:39.873005+0000","last_undegraded":"2019-11-25T09:15:39.873005+0000","last_fullsized":"2019-11-25T09:15:39.873005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38344,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.685}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.561}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.554}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.071}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.658}]}]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":47,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48313,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.847}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.881}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":43.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.244}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.419}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.009}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.960}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38691,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.859}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.912}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.315}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.390}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.253}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.879}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.394}]}]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":33,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33915,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.208}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.134}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.907}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.896}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.780}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.422}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.889}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38773,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.538}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.661}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.415}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.858}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.718}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.633}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.797}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":33,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33871,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.363}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.453}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.647}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.092}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.397}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.258}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.634}]}]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38729,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.043}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.587}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.751}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.702}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.634}]}]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43568,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.773}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.893}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.359}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.215}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.179}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.991}]}]}]}} 2019-11-25T09:15:55.620 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2019-11-25T09:15:56.519 INFO:teuthology.orchestra.run.smithi153.stderr:dumped all 2019-11-25T09:15:56.520 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:15:56.539 INFO:teuthology.orchestra.run.smithi153.stdout:{"pg_ready":true,"pg_map":{"version":35,"stamp":"2019-11-25T09:15:55.372706+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":16,"num_osds":8,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":749595040,"kb_used":1699648,"kb_used_data":747895392,"kb_used_omap":306,"kb_used_meta":0,"kb_avail":747895392,"statfs":{"total":767585320960,"available":765844881408,"internally_reserved":0,"allocated":765844881408,"data_stored":765844881408,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":314204,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.258289+0000","last_change":"2019-11-25T09:15:39.326784+0000","last_active":"2019-11-25T09:15:40.258289+0000","last_peered":"2019-11-25T09:15:40.258289+0000","last_clean":"2019-11-25T09:15:40.258289+0000","last_became_active":"2019-11-25T09:15:39.293677+0000","last_became_peered":"2019-11-25T09:15:39.293677+0000","last_unstale":"2019-11-25T09:15:40.258289+0000","last_undegraded":"2019-11-25T09:15:40.258289+0000","last_fullsized":"2019-11-25T09:15:40.258289+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,6],"acting":[1,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.497509+0000","last_change":"2019-11-25T09:15:40.475676+0000","last_active":"2019-11-25T09:15:40.497509+0000","last_peered":"2019-11-25T09:15:40.497509+0000","last_clean":"2019-11-25T09:15:40.497509+0000","last_became_active":"2019-11-25T09:15:40.409422+0000","last_became_peered":"2019-11-25T09:15:40.409422+0000","last_unstale":"2019-11-25T09:15:40.497509+0000","last_undegraded":"2019-11-25T09:15:40.497509+0000","last_fullsized":"2019-11-25T09:15:40.497509+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.263681+0000","last_change":"2019-11-25T09:15:39.364197+0000","last_active":"2019-11-25T09:15:40.263681+0000","last_peered":"2019-11-25T09:15:40.263681+0000","last_clean":"2019-11-25T09:15:40.263681+0000","last_became_active":"2019-11-25T09:15:39.310975+0000","last_became_peered":"2019-11-25T09:15:39.310975+0000","last_unstale":"2019-11-25T09:15:40.263681+0000","last_undegraded":"2019-11-25T09:15:40.263681+0000","last_fullsized":"2019-11-25T09:15:40.263681+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,6],"acting":[2,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.266381+0000","last_change":"2019-11-25T09:15:39.082760+0000","last_active":"2019-11-25T09:15:40.266381+0000","last_peered":"2019-11-25T09:15:40.266381+0000","last_clean":"2019-11-25T09:15:40.266381+0000","last_became_active":"2019-11-25T09:15:39.032784+0000","last_became_peered":"2019-11-25T09:15:39.032784+0000","last_unstale":"2019-11-25T09:15:40.266381+0000","last_undegraded":"2019-11-25T09:15:40.266381+0000","last_fullsized":"2019-11-25T09:15:40.266381+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[7,0],"acting":[7,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":7,"acting_primary":7,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.250971+0000","last_change":"2019-11-25T09:15:39.388305+0000","last_active":"2019-11-25T09:15:40.250971+0000","last_peered":"2019-11-25T09:15:40.250971+0000","last_clean":"2019-11-25T09:15:40.250971+0000","last_became_active":"2019-11-25T09:15:39.375469+0000","last_became_peered":"2019-11-25T09:15:39.375469+0000","last_unstale":"2019-11-25T09:15:40.250971+0000","last_undegraded":"2019-11-25T09:15:40.250971+0000","last_fullsized":"2019-11-25T09:15:40.250971+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,5],"acting":[1,5],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:40.492415+0000","last_change":"2019-11-25T09:15:40.462512+0000","last_active":"2019-11-25T09:15:40.492415+0000","last_peered":"2019-11-25T09:15:40.492415+0000","last_clean":"2019-11-25T09:15:40.492415+0000","last_became_active":"2019-11-25T09:15:40.381235+0000","last_became_peered":"2019-11-25T09:15:40.381235+0000","last_unstale":"2019-11-25T09:15:40.492415+0000","last_undegraded":"2019-11-25T09:15:40.492415+0000","last_fullsized":"2019-11-25T09:15:40.492415+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":19,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,6],"acting":[3,6],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:39.913253+0000","last_change":"2019-11-25T09:15:39.113495+0000","last_active":"2019-11-25T09:15:39.913253+0000","last_peered":"2019-11-25T09:15:39.913253+0000","last_clean":"2019-11-25T09:15:39.913253+0000","last_became_active":"2019-11-25T09:15:39.089643+0000","last_became_peered":"2019-11-25T09:15:39.089643+0000","last_unstale":"2019-11-25T09:15:39.913253+0000","last_undegraded":"2019-11-25T09:15:39.913253+0000","last_fullsized":"2019-11-25T09:15:39.913253+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,4],"acting":[0,4],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"0'0","reported_seq":"10","reported_epoch":"19","state":"active+clean","last_fresh":"2019-11-25T09:15:39.873005+0000","last_change":"2019-11-25T09:15:39.134492+0000","last_active":"2019-11-25T09:15:39.873005+0000","last_peered":"2019-11-25T09:15:39.873005+0000","last_clean":"2019-11-25T09:15:39.873005+0000","last_became_active":"2019-11-25T09:15:39.091359+0000","last_became_peered":"2019-11-25T09:15:39.091359+0000","last_unstale":"2019-11-25T09:15:39.873005+0000","last_undegraded":"2019-11-25T09:15:39.873005+0000","last_fullsized":"2019-11-25T09:15:39.873005+0000","mapping_epoch":17,"log_start":"0'0","ondisk_log_start":"0'0","created":17,"last_epoch_clean":18,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2019-11-25T09:15:36.915481+0000","last_clean_scrub_stamp":"2019-11-25T09:15:36.915481+0000","log_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[4,1],"acting":[4,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":4,"acting_primary":4,"purged_snaps":[]}],"pool_stats":[{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":16,"acting":16,"num_store_stats":0}],"osd_stats":[{"osd":7,"up_from":15,"seq":64424509449,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38344,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,6],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.685}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.561}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.117}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.554}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.071}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":40.747}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.658}]}]},{"osd":6,"up_from":15,"seq":64424509449,"num_pgs":4,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":47,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":48313,"internal_metadata":0},"hb_peers":[0,1,2,3,4,5,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.847}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":31.881}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":43.161}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.244}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.419}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.009}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.960}]}]},{"osd":4,"up_from":15,"seq":64424509448,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38691,"internal_metadata":0},"hb_peers":[0,1,2,3,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":18.859}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.912}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.315}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":32.390}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":23.253}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.879}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.394}]}]},{"osd":2,"up_from":15,"seq":64424509449,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":33,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33915,"internal_metadata":0},"hb_peers":[0,1,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.208}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":37.134}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.907}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.896}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.780}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":38.422}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.889}]}]},{"osd":3,"up_from":14,"seq":60129542152,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38773,"internal_metadata":0},"hb_peers":[0,1,2,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.538}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":12.661}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":13.415}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.858}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":10.718}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":11.633}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.797}]}]},{"osd":5,"up_from":13,"seq":55834574857,"num_pgs":1,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":33,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":33871,"internal_metadata":0},"hb_peers":[0,1,2,3,4,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.363}]},{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.453}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":24.647}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":21.092}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":41.397}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.258}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":35.634}]}]},{"osd":0,"up_from":15,"seq":64424509449,"num_pgs":2,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":37,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":38729,"internal_metadata":0},"hb_peers":[1,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":1,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":28.307}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":25.043}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":29.111}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":15.587}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.751}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.702}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":20.634}]}]},{"osd":1,"up_from":15,"seq":64424509449,"num_pgs":3,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":93699380,"kb_used":212456,"kb_used_data":93486924,"kb_used_omap":42,"kb_used_meta":0,"kb_avail":93486924,"statfs":{"total":95948165120,"available":95730610176,"internally_reserved":0,"allocated":95730610176,"data_stored":95730610176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":43568,"internal_metadata":0},"hb_peers":[0,2,3,4,5,6,7],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[{"osd":0,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":14.773}]},{"osd":2,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":22.893}]},{"osd":3,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":16.359}]},{"osd":4,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":17.215}]},{"osd":5,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":26.504}]},{"osd":6,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":19.179}]},{"osd":7,"last update":"Thu Jan 1 00:00:00 1970","interfaces":[{"interface":"back","average":{"1min":0.000,"5min":0.000,"15min":0.000},"min":{"1min":0.000,"5min":0.000,"15min":0.000},"max":{"1min":0.000,"5min":0.000,"15min":0.000},"last":27.991}]}]}]}} 2019-11-25T09:15:56.542 INFO:tasks.ceph.ceph_manager.ceph:clean! 2019-11-25T09:15:56.542 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2019-11-25T09:15:56.542 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2019-11-25T09:15:56.542 INFO:teuthology.orchestra.run.smithi153:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2019-11-25T09:15:57.457 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:15:57.478 INFO:teuthology.orchestra.run.smithi153.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2019-11-25T09:15:57.478 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2019-11-25T09:15:57.479 INFO:teuthology.run_tasks:Running task openssl_keys... 2019-11-25T09:15:57.493 INFO:teuthology.run_tasks:Running task rgw... 2019-11-25T09:15:57.515 DEBUG:tasks.rgw:config is {'client.0': {'valgrind': ['--tool=memcheck', '--max-threads=1024']}} 2019-11-25T09:15:57.515 DEBUG:tasks.rgw:client list is ['client.0'] 2019-11-25T09:15:57.516 INFO:tasks.rgw:Creating data pools 2019-11-25T09:15:57.516 DEBUG:tasks.rgw:Obtaining remote for client client.0 2019-11-25T09:15:57.516 INFO:teuthology.orchestra.run.smithi153:> sudo ceph osd erasure-code-profile set client.0 crush-failure-domain=osd k=2 m=1 --cluster ceph 2019-11-25T09:15:58.924 INFO:teuthology.orchestra.run.smithi153:> sudo ceph osd pool create default.rgw.buckets.data 64 64 erasure client.0 --cluster ceph 2019-11-25T09:16:01.456 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'default.rgw.buckets.data' created 2019-11-25T09:16:01.473 INFO:teuthology.orchestra.run.smithi153:> sudo ceph osd pool application enable default.rgw.buckets.data rgw --cluster ceph 2019-11-25T09:16:02.442 INFO:teuthology.orchestra.run.smithi153.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.data' 2019-11-25T09:16:02.458 INFO:teuthology.orchestra.run.smithi153:> sudo ceph osd pool create default.rgw.buckets.index 64 64 --cluster ceph 2019-11-25T09:16:03.785 INFO:teuthology.orchestra.run.smithi153.stderr:pool 'default.rgw.buckets.index' created 2019-11-25T09:16:03.804 INFO:teuthology.orchestra.run.smithi153:> sudo ceph osd pool application enable default.rgw.buckets.index rgw --cluster ceph 2019-11-25T09:16:04.861 INFO:teuthology.orchestra.run.smithi153.stderr:enabled application 'rgw' on pool 'default.rgw.buckets.index' 2019-11-25T09:16:04.877 DEBUG:tasks.rgw:Pools created 2019-11-25T09:16:04.877 INFO:tasks.rgw:Configuring compression type = random 2019-11-25T09:16:04.877 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-11-25T09:16:04.878 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-11-25T09:16:04.878 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-11-25T09:16:05.063 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:05.065+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:07.854 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:07.855+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:07.854 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:07.855+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:07.864+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:07.863 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:07.864+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:07.874 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:07.875+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:07.874 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:07.875+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:07.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:07.992+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:09.321 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.322+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:09.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.341+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:09.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.341+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:09.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.351+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:09.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.351+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:09.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.358+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:09.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.358+0000 7f074fbe3ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:16:09.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.358+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:09.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.366+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:09.366 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.366+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:09.376 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.378+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:09.376 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.378+0000 7f074fbe3ec0 10 Creating default zonegroup 2019-11-25T09:16:09.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.378+0000 7f074fbe3ec0 20 get_system_obj_state: rctx=0x7ffcdad7f450 obj=:.pools.avail state=0x557f83104ab0 s->prefetch_data=0 2019-11-25T09:16:09.381 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.382+0000 7f074fbe3ec0 10 couldn't find old data placement pools config, setting up new ones for the zone 2019-11-25T09:16:09.590 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.591+0000 7f074fbe3ec0 10 WARNING: store->list_zones() returned r=-2 2019-11-25T09:16:09.590 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.591+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:09.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:09.601+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:10.669 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:10.668+0000 7f073ce11700 0 -- 172.21.15.153:0/881020078 >> [v2:172.21.15.81:6808/9717,v1:172.21.15.81:6809/9717] conn(0x557f830fd910 msgr2=0x557f830ffd90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:16:10.673 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:10.668+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:10.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:10.680+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:10.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:10.680+0000 7f074fbe3ec0 10 could not read realm id: (2) No such file or directory 2019-11-25T09:16:10.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:10.680+0000 7f074fbe3ec0 10 WARNING: failed to set zone as default, r=-22 2019-11-25T09:16:10.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:10.680+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:10.689 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:10.691+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:11.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.246+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:11.313 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.310+0000 7f074fbe3ec0 20 rados_obj.operate() r=0 bl.length=805 2019-11-25T09:16:11.313 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.310+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:11.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.372+0000 7f074fbe3ec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-25T09:16:11.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.372+0000 7f074fbe3ec0 20 zonegroup default 2019-11-25T09:16:11.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.372+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.386+0000 7f074fbe3ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.386+0000 7f074fbe3ec0 10 Cannot find current period zone using local zone 2019-11-25T09:16:11.387 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.386+0000 7f074fbe3ec0 10 Using default name default 2019-11-25T09:16:11.388 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.386+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:11.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.443+0000 7f074fbe3ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:11.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.443+0000 7f074fbe3ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:11.462 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.463+0000 7f074fbe3ec0 20 rados_obj.operate() r=0 bl.length=805 2019-11-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.463+0000 7f074fbe3ec0 20 zone default found 2019-11-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.463+0000 7f074fbe3ec0 20 started sync module instance, tier type = 2019-11-25T09:16:11.463 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:11.463+0000 7f074fbe3ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:16:15.622 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:15.623+0000 7f0737fff700 0 -- 172.21.15.153:0/881020078 >> [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] conn(0x557f83101ff0 msgr2=0x557f83104410 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T09:16:17.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.438+0000 7f074fbe3ec0 20 add_watcher() i=0 2019-11-25T09:16:17.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.438+0000 7f074fbe3ec0 20 add_watcher() i=1 2019-11-25T09:16:17.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.438+0000 7f074fbe3ec0 20 add_watcher() i=2 2019-11-25T09:16:17.438 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.438+0000 7f074fbe3ec0 20 add_watcher() i=3 2019-11-25T09:16:17.438 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.438+0000 7f074fbe3ec0 20 add_watcher() i=4 2019-11-25T09:16:17.438 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.438+0000 7f074fbe3ec0 20 add_watcher() i=5 2019-11-25T09:16:17.457 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.458+0000 7f074fbe3ec0 20 add_watcher() i=6 2019-11-25T09:16:17.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.483+0000 7f074fbe3ec0 20 add_watcher() i=7 2019-11-25T09:16:17.483 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.483+0000 7f074fbe3ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:16:17.483 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:17.484+0000 7f070cfe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:16:21.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:21.303+0000 7f074fbe3ec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.meta 2019-11-25T09:16:26.047 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:26.048+0000 7f074fbe3ec0 10 rgw_init_ioctx warning: failed to set recovery_priority on default.rgw.log 2019-11-25T09:16:29.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:29.711+0000 7f067affd700 20 reqs_thread_entry: start 2019-11-25T09:16:29.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:29.711+0000 7f0678ff9700 20 reqs_thread_entry: start 2019-11-25T09:16:29.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:29.711+0000 7f074fbe3ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:16:29.711 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:29.712+0000 7f0666ffd700 20 reqs_thread_entry: start 2019-11-25T09:16:29.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:29.985+0000 7f074fbe3ec0 20 remove_watcher() i=0 2019-11-25T09:16:29.991 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:29.985+0000 7f074fbe3ec0 2 removed watcher, disabling cache 2019-11-25T09:16:30.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.082+0000 7f074fbe3ec0 20 remove_watcher() i=1 2019-11-25T09:16:30.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.168+0000 7f074fbe3ec0 20 remove_watcher() i=2 2019-11-25T09:16:30.263 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.264+0000 7f074fbe3ec0 20 remove_watcher() i=3 2019-11-25T09:16:30.316 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.318+0000 7f074fbe3ec0 20 remove_watcher() i=4 2019-11-25T09:16:30.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.372+0000 7f074fbe3ec0 20 remove_watcher() i=5 2019-11-25T09:16:30.519 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.521+0000 7f074fbe3ec0 20 remove_watcher() i=6 2019-11-25T09:16:30.582 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.584+0000 7f074fbe3ec0 20 remove_watcher() i=7 2019-11-25T09:16:30.592 INFO:teuthology.orchestra.run.smithi153.stdout:[] 2019-11-25T09:16:30.592 DEBUG:tasks.util.rgw: json result: [] 2019-11-25T09:16:30.593 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-11-25T09:16:30.593 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'modify', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--compression', 'random'] 2019-11-25T09:16:30.593 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement modify --rgw-zone default --placement-id default-placement --compression random 2019-11-25T09:16:30.788 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.789+0000 7f8f2f7fe700 20 reqs_thread_entry: start 2019-11-25T09:16:30.789 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.789+0000 7f8ffa841ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:30.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.810+0000 7f8ffa841ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:30.810 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.810+0000 7f8ffa841ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:30.820 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.822+0000 7f8ffa841ec0 20 rados_obj.operate() r=0 bl.length=805 2019-11-25T09:16:30.821 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.822+0000 7f8ffa841ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:30.838 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.839+0000 7f8ffa841ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:30.838 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.839+0000 7f8ffa841ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:30.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.852+0000 7f8ffa841ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:30.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.852+0000 7f8ffa841ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:30.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:30.871+0000 7f8ffa841ec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-25T09:16:31.035 INFO:teuthology.orchestra.run.smithi153.stdout:{"id":"103c5a94-eb69-4977-8c90-9f6d634d8761","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-11-25T09:16:31.036 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'103c5a94-eb69-4977-8c90-9f6d634d8761', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-11-25T09:16:31.036 INFO:tasks.util.rgw:rgwadmin: client.0 : ['user', 'list'] 2019-11-25T09:16:31.036 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'user', 'list'] 2019-11-25T09:16:31.036 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph user list 2019-11-25T09:16:31.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.187+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.218 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.219+0000 7f409ebd4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:31.218 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.220+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.231+0000 7f409ebd4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:31.230 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.231+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.246+0000 7f409ebd4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:31.245 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.246+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.265 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.267+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:31.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.602+0000 7f409ebd4ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:16:31.601 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.602+0000 7f409ebd4ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:16:31.601 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.602+0000 7f409ebd4ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:16:31.601 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.602+0000 7f409ebd4ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:16:31.601 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.602+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.620 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.621+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:31.620 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.622+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.640 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.641+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=821 2019-11-25T09:16:31.640 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.641+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.658 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.660+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:31.659 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.660+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.677 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.679+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-25T09:16:31.678 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.679+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.686 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.688+0000 7f409ebd4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:31.686 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.688+0000 7f409ebd4ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:16:31.686 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.688+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.697+0000 7f409ebd4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:31.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.697+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.712+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:31.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.712+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.730+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-25T09:16:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.730+0000 7f409ebd4ec0 20 zonegroup default 2019-11-25T09:16:31.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.730+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.742 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.741+0000 7f409ebd4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:31.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.741+0000 7f409ebd4ec0 10 Cannot find current period zone using local zone 2019-11-25T09:16:31.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.741+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.761 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.750+0000 7f409ebd4ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:31.761 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.750+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.761 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.762+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:31.761 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.762+0000 7f409ebd4ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:31.780 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.781+0000 7f409ebd4ec0 20 rados_obj.operate() r=0 bl.length=821 2019-11-25T09:16:31.781 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.781+0000 7f409ebd4ec0 20 zone default found 2019-11-25T09:16:31.781 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.781+0000 7f409ebd4ec0 20 started sync module instance, tier type = 2019-11-25T09:16:31.781 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:31.782+0000 7f409ebd4ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:16:32.245 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.247+0000 7f409ebd4ec0 20 add_watcher() i=0 2019-11-25T09:16:32.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.247+0000 7f409ebd4ec0 20 add_watcher() i=1 2019-11-25T09:16:32.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.247+0000 7f409ebd4ec0 20 add_watcher() i=2 2019-11-25T09:16:32.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.247+0000 7f409ebd4ec0 20 add_watcher() i=3 2019-11-25T09:16:32.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.247+0000 7f409ebd4ec0 20 add_watcher() i=4 2019-11-25T09:16:32.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.247+0000 7f409ebd4ec0 20 add_watcher() i=5 2019-11-25T09:16:32.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.247+0000 7f409ebd4ec0 20 add_watcher() i=6 2019-11-25T09:16:32.272 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.274+0000 7f409ebd4ec0 20 add_watcher() i=7 2019-11-25T09:16:32.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.274+0000 7f409ebd4ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:16:32.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.274+0000 7f3fdffff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:16:32.281 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.282+0000 7f408be02700 0 -- 172.21.15.153:0/783909860 >> v2:172.21.15.81:3300/0 conn(0x55fe6c76ff70 msgr2=0x55fe6c776850 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:16:32.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.486+0000 7f408be02700 0 --2- 172.21.15.153:0/783909860 >> v2:172.21.15.81:3300/0 conn(0x55fe6c76ff70 0x55fe6c776850 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-25T09:16:32.558 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:32.560+0000 7f408be02700 0 -- 172.21.15.153:0/1773720025 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7f3fcc017580 msgr2=0x7f3fcc019a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T09:16:35.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.119+0000 7f3fca7fc700 20 reqs_thread_entry: start 2019-11-25T09:16:35.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.119+0000 7f3fc3fff700 20 reqs_thread_entry: start 2019-11-25T09:16:35.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.119+0000 7f409ebd4ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:16:35.119 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.120+0000 7f3fc27fc700 20 reqs_thread_entry: start 2019-11-25T09:16:35.329 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.328+0000 7f409ebd4ec0 20 remove_watcher() i=0 2019-11-25T09:16:35.329 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.328+0000 7f409ebd4ec0 2 removed watcher, disabling cache 2019-11-25T09:16:35.384 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.385+0000 7f409ebd4ec0 20 remove_watcher() i=1 2019-11-25T09:16:35.460 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.462+0000 7f409ebd4ec0 20 remove_watcher() i=2 2019-11-25T09:16:35.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.519+0000 7f409ebd4ec0 20 remove_watcher() i=3 2019-11-25T09:16:35.569 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.567+0000 7f409ebd4ec0 20 remove_watcher() i=4 2019-11-25T09:16:35.607 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.609+0000 7f409ebd4ec0 20 remove_watcher() i=5 2019-11-25T09:16:35.671 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.672+0000 7f409ebd4ec0 20 remove_watcher() i=6 2019-11-25T09:16:35.736 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.738+0000 7f409ebd4ec0 20 remove_watcher() i=7 2019-11-25T09:16:35.746 INFO:teuthology.orchestra.run.smithi153.stdout:[] 2019-11-25T09:16:35.747 DEBUG:tasks.util.rgw: json result: [] 2019-11-25T09:16:35.747 INFO:tasks.rgw:Configuring storage class type = LUKEWARM 2019-11-25T09:16:35.747 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-11-25T09:16:35.747 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM'] 2019-11-25T09:16:35.747 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM 2019-11-25T09:16:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.962+0000 7f9bc2ffd700 20 reqs_thread_entry: start 2019-11-25T09:16:35.961 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.962+0000 7f9c8da61ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:35.980 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.982+0000 7f9c8da61ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:35.981 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:35.982+0000 7f9c8da61ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.003+0000 7f9c8da61ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:36.006 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.003+0000 7f9c8da61ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.014 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.015+0000 7f9c8da61ec0 20 rados_obj.operate() r=0 bl.length=348 2019-11-25T09:16:36.014 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.015+0000 7f9c8da61ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.026 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.027+0000 7f9c8da61ec0 20 rados_obj.operate() r=0 bl.length=821 2019-11-25T09:16:36.117 INFO:teuthology.orchestra.run.smithi153.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["LUKEWARM","STANDARD"]}}] 2019-11-25T09:16:36.117 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-11-25T09:16:36.118 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-11-25T09:16:36.118 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'LUKEWARM', '--data-pool', 'default.rgw.buckets.data.lukewarm'] 2019-11-25T09:16:36.118 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class LUKEWARM --data-pool default.rgw.buckets.data.lukewarm 2019-11-25T09:16:36.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.406+0000 7f80ee7fc700 20 reqs_thread_entry: start 2019-11-25T09:16:36.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.406+0000 7f81b94f5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.428+0000 7f81b94f5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:36.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.428+0000 7f81b94f5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.445 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.443+0000 7f81b94f5ec0 20 rados_obj.operate() r=0 bl.length=821 2019-11-25T09:16:36.445 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.443+0000 7f81b94f5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.457 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.459+0000 7f81b94f5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:36.458 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.459+0000 7f81b94f5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.473 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.475+0000 7f81b94f5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:36.474 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.475+0000 7f81b94f5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.497 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.498+0000 7f81b94f5ec0 20 rados_obj.operate() r=0 bl.length=372 2019-11-25T09:16:36.583 INFO:teuthology.orchestra.run.smithi153.stdout:{"id":"103c5a94-eb69-4977-8c90-9f6d634d8761","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-11-25T09:16:36.584 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'103c5a94-eb69-4977-8c90-9f6d634d8761', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-11-25T09:16:36.585 INFO:tasks.rgw:Configuring storage class type = FROZEN 2019-11-25T09:16:36.585 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-11-25T09:16:36.585 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zonegroup', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN'] 2019-11-25T09:16:36.586 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zonegroup placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN 2019-11-25T09:16:36.846 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.847+0000 7fc7db7fe700 20 reqs_thread_entry: start 2019-11-25T09:16:36.846 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.847+0000 7fc8a629bec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.864 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.865+0000 7fc8a629bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:36.864 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.865+0000 7fc8a629bec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.900 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.900+0000 7fc8a629bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:36.900 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.900+0000 7fc8a629bec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.921 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.923+0000 7fc8a629bec0 20 rados_obj.operate() r=0 bl.length=372 2019-11-25T09:16:36.921 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.923+0000 7fc8a629bec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:36.953 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:36.955+0000 7fc8a629bec0 20 rados_obj.operate() r=0 bl.length=888 2019-11-25T09:16:37.024 INFO:teuthology.orchestra.run.smithi153.stdout:[{"key":"default-placement","val":{"name":"default-placement","tags":[],"storage_classes":["FROZEN","LUKEWARM","STANDARD"]}}] 2019-11-25T09:16:37.024 DEBUG:tasks.util.rgw: json result: [{u'val': {u'storage_classes': [u'FROZEN', u'LUKEWARM', u'STANDARD'], u'name': u'default-placement', u'tags': []}, u'key': u'default-placement'}] 2019-11-25T09:16:37.025 INFO:tasks.util.rgw:rgwadmin: client.0 : ['zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-11-25T09:16:37.025 DEBUG:tasks.util.rgw:rgwadmin: cmd=['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '--log-to-stderr', '--format', 'json', '-n', 'client.0', '--cluster', 'ceph', 'zone', 'placement', 'add', '--rgw-zone', 'default', '--placement-id', 'default-placement', '--storage-class', 'FROZEN', '--data-pool', 'default.rgw.buckets.data.frozen'] 2019-11-25T09:16:37.025 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin --log-to-stderr --format json -n client.0 --cluster ceph zone placement add --rgw-zone default --placement-id default-placement --storage-class FROZEN --data-pool default.rgw.buckets.data.frozen 2019-11-25T09:16:37.179 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.181+0000 7f8137fff700 20 reqs_thread_entry: start 2019-11-25T09:16:37.180 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.181+0000 7f8204051ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:37.192 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.193+0000 7f8204051ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:37.192 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.193+0000 7f8204051ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:37.204 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.205+0000 7f8204051ec0 20 rados_obj.operate() r=0 bl.length=888 2019-11-25T09:16:37.205 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.205+0000 7f8204051ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:37.217 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.218+0000 7f8204051ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:16:37.217 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.218+0000 7f8204051ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:37.225 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.227+0000 7f8204051ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:16:37.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.227+0000 7f8204051ec0 20 rados->read ofs=0 len=0 2019-11-25T09:16:37.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:16:37.238+0000 7f8204051ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:16:37.301 INFO:teuthology.orchestra.run.smithi153.stdout:{"id":"103c5a94-eb69-4977-8c90-9f6d634d8761","name":"default","domain_root":"default.rgw.meta:root","control_pool":"default.rgw.control","gc_pool":"default.rgw.log:gc","lc_pool":"default.rgw.log:lc","log_pool":"default.rgw.log","intent_log_pool":"default.rgw.log:intent","usage_log_pool":"default.rgw.log:usage","roles_pool":"default.rgw.meta:roles","reshard_pool":"default.rgw.log:reshard","user_keys_pool":"default.rgw.meta:users.keys","user_email_pool":"default.rgw.meta:users.email","user_swift_pool":"default.rgw.meta:users.swift","user_uid_pool":"default.rgw.meta:users.uid","otp_pool":"default.rgw.otp","system_key":{"access_key":"","secret_key":""},"placement_pools":[{"key":"default-placement","val":{"index_pool":"default.rgw.buckets.index","storage_classes":{"FROZEN":{"data_pool":"default.rgw.buckets.data.frozen"},"LUKEWARM":{"data_pool":"default.rgw.buckets.data.lukewarm"},"STANDARD":{"data_pool":"default.rgw.buckets.data","compression_type":"random"}},"data_extra_pool":"default.rgw.buckets.non-ec","index_type":0}}],"realm_id":""} 2019-11-25T09:16:37.302 DEBUG:tasks.util.rgw: json result: {u'control_pool': u'default.rgw.control', u'otp_pool': u'default.rgw.otp', u'usage_log_pool': u'default.rgw.log:usage', u'name': u'default', u'user_uid_pool': u'default.rgw.meta:users.uid', u'gc_pool': u'default.rgw.log:gc', u'intent_log_pool': u'default.rgw.log:intent', u'roles_pool': u'default.rgw.meta:roles', u'lc_pool': u'default.rgw.log:lc', u'user_keys_pool': u'default.rgw.meta:users.keys', u'domain_root': u'default.rgw.meta:root', u'system_key': {u'access_key': u'', u'secret_key': u''}, u'reshard_pool': u'default.rgw.log:reshard', u'log_pool': u'default.rgw.log', u'user_email_pool': u'default.rgw.meta:users.email', u'realm_id': u'', u'placement_pools': [{u'val': {u'storage_classes': {u'FROZEN': {u'data_pool': u'default.rgw.buckets.data.frozen'}, u'LUKEWARM': {u'data_pool': u'default.rgw.buckets.data.lukewarm'}, u'STANDARD': {u'data_pool': u'default.rgw.buckets.data', u'compression_type': u'random'}}, u'data_extra_pool': u'default.rgw.buckets.non-ec', u'index_type': 0, u'index_pool': u'default.rgw.buckets.index'}, u'key': u'default-placement'}], u'id': u'103c5a94-eb69-4977-8c90-9f6d634d8761', u'user_swift_pool': u'default.rgw.meta:users.swift'} 2019-11-25T09:16:37.302 INFO:tasks.rgw:Starting rgw... 2019-11-25T09:16:37.302 INFO:tasks.rgw:rgw client.0 config is {'valgrind': ['--tool=memcheck', '--max-threads=1024']} 2019-11-25T09:16:37.302 INFO:tasks.rgw:Using civetweb as radosgw frontend 2019-11-25T09:16:37.303 DEBUG:teuthology.misc:running ceph.client.0 under valgrind with args ['cd', '/home/ubuntu/cephtest', Raw('&&'), 'sudo', 'adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'daemon-helper', 'term', 'valgrind', '--trace-children=no', '--child-silent-after-fork=yes', '--num-callers=50', '--suppressions=/home/ubuntu/cephtest/valgrind.supp', '--xml=yes', '--xml-file=/var/log/ceph/valgrind/ceph.client.0.log', '--time-stamp=yes', '--tool=memcheck', '--max-threads=1024'] 2019-11-25T09:16:37.303 INFO:tasks.rgw.client.0:Restarting daemon 2019-11-25T09:16:37.303 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper term valgrind --trace-children=no --child-silent-after-fork=yes --num-callers=50 --suppressions=/home/ubuntu/cephtest/valgrind.supp --xml=yes --xml-file=/var/log/ceph/valgrind/ceph.client.0.log --time-stamp=yes --tool=memcheck --max-threads=1024 radosgw --rgw-frontends 'civetweb port=80' -n client.0 --cluster ceph -k /etc/ceph/ceph.client.0.keyring --log-file /var/log/ceph/rgw.ceph.client.0.log --rgw_ops_log_socket_path /home/ubuntu/cephtest/rgw.opslog.ceph.client.0.sock --foreground | sudo tee /var/log/ceph/rgw.ceph.client.0.stdout 2>&1 2019-11-25T09:16:37.312 INFO:tasks.rgw.client.0:Started 2019-11-25T09:16:37.313 INFO:tasks.rgw:Polling client.0 until it starts accepting connections on http://smithi153.front.sepia.ceph.com:80/ 2019-11-25T09:16:37.313 INFO:teuthology.orchestra.run.smithi153:> curl http://smithi153.front.sepia.ceph.com:80/ 2019-11-25T09:16:37.390 INFO:teuthology.orchestra.run.smithi153.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-25T09:16:37.391 INFO:teuthology.orchestra.run.smithi153.stderr: Dload Upload Total Spent Left Speed 2019-11-25T09:16:37.394 INFO:teuthology.orchestra.run.smithi153.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi153.front.sepia.ceph.com:80; Connection refused 2019-11-25T09:16:37.394 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-11-25T09:16:38.395 INFO:teuthology.orchestra.run.smithi153:> curl http://smithi153.front.sepia.ceph.com:80/ 2019-11-25T09:16:38.485 INFO:teuthology.orchestra.run.smithi153.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-25T09:16:38.487 INFO:teuthology.orchestra.run.smithi153.stderr: Dload Upload Total Spent Left Speed 2019-11-25T09:16:38.487 INFO:teuthology.orchestra.run.smithi153.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi153.front.sepia.ceph.com:80; Connection refused 2019-11-25T09:16:38.487 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-11-25T09:16:40.488 INFO:teuthology.orchestra.run.smithi153:> curl http://smithi153.front.sepia.ceph.com:80/ 2019-11-25T09:16:40.574 INFO:teuthology.orchestra.run.smithi153.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-25T09:16:40.579 INFO:teuthology.orchestra.run.smithi153.stderr: Dload Upload Total Spent Left Speed 2019-11-25T09:16:40.579 INFO:teuthology.orchestra.run.smithi153.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi153.front.sepia.ceph.com:80; Connection refused 2019-11-25T09:16:40.580 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-11-25T09:16:44.580 INFO:teuthology.orchestra.run.smithi153:> curl http://smithi153.front.sepia.ceph.com:80/ 2019-11-25T09:16:44.678 INFO:teuthology.orchestra.run.smithi153.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-25T09:16:44.680 INFO:teuthology.orchestra.run.smithi153.stderr: Dload Upload Total Spent Left Speed 2019-11-25T09:16:44.683 INFO:teuthology.orchestra.run.smithi153.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0curl: (7) Failed connect to smithi153.front.sepia.ceph.com:80; Connection refused 2019-11-25T09:16:44.683 DEBUG:teuthology.orchestra.run:got remote process result: 7 2019-11-25T09:16:52.685 INFO:teuthology.orchestra.run.smithi153:> curl http://smithi153.front.sepia.ceph.com:80/ 2019-11-25T09:16:52.797 INFO:teuthology.orchestra.run.smithi153.stderr: % Total % Received % Xferd Average Speed Time Time Time Current 2019-11-25T09:16:52.800 INFO:teuthology.orchestra.run.smithi153.stderr: Dload Upload Total Spent Left Speed 2019-11-25T09:16:53.198 INFO:teuthology.orchestra.run.smithi153.stderr: 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 214 0 214 0 0 490 0 --:--:-- --:--:-- --:--:-- 489 2019-11-25T09:16:53.199 INFO:teuthology.orchestra.run.smithi153.stdout:anonymous 2019-11-25T09:16:53.199 INFO:teuthology.run_tasks:Running task workunit... 2019-11-25T09:16:53.216 INFO:tasks.workunit:Pulling workunits from ref 12b9659be24ae0e3d791f80d81504ab2770c1343 2019-11-25T09:16:53.218 INFO:tasks.workunit:Making a separate scratch dir for every client... 2019-11-25T09:16:53.218 INFO:teuthology.orchestra.run.smithi153:> stat -- /home/ubuntu/cephtest/mnt.0 2019-11-25T09:16:53.273 INFO:teuthology.orchestra.run.smithi153.stderr:stat: cannot stat ‘/home/ubuntu/cephtest/mnt.0’: No such file or directory 2019-11-25T09:16:53.273 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-11-25T09:16:53.274 INFO:teuthology.orchestra.run.smithi153:> mkdir -- /home/ubuntu/cephtest/mnt.0 2019-11-25T09:16:53.317 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2019-11-25T09:16:53.318 INFO:teuthology.orchestra.run.smithi153:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2019-11-25T09:16:53.451 INFO:tasks.workunit:timeout=3h 2019-11-25T09:16:53.452 INFO:tasks.workunit:cleanup=True 2019-11-25T09:16:53.452 INFO:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone git://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 12b9659be24ae0e3d791f80d81504ab2770c1343 2019-11-25T09:16:53.737 INFO:tasks.workunit.client.0.smithi153.stdout:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2019-11-25T09:18:00.139 INFO:tasks.workunit.client.0.smithi153.stderr:Note: checking out '12b9659be24ae0e3d791f80d81504ab2770c1343'. 2019-11-25T09:18:00.140 INFO:tasks.workunit.client.0.smithi153.stderr: 2019-11-25T09:18:00.140 INFO:tasks.workunit.client.0.smithi153.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2019-11-25T09:18:00.140 INFO:tasks.workunit.client.0.smithi153.stderr:changes and commit them, and you can discard any commits you make in this 2019-11-25T09:18:00.140 INFO:tasks.workunit.client.0.smithi153.stderr:state without impacting any branches by performing another checkout. 2019-11-25T09:18:00.140 INFO:tasks.workunit.client.0.smithi153.stderr: 2019-11-25T09:18:00.140 INFO:tasks.workunit.client.0.smithi153.stderr:If you want to create a new branch to retain commits you create, you may 2019-11-25T09:18:00.141 INFO:tasks.workunit.client.0.smithi153.stderr:do so (now or later) by using -b with the checkout command again. Example: 2019-11-25T09:18:00.141 INFO:tasks.workunit.client.0.smithi153.stderr: 2019-11-25T09:18:00.141 INFO:tasks.workunit.client.0.smithi153.stderr: git checkout -b new_branch_name 2019-11-25T09:18:00.141 INFO:tasks.workunit.client.0.smithi153.stderr: 2019-11-25T09:18:00.141 INFO:tasks.workunit.client.0.smithi153.stderr:HEAD is now at 12b9659... rgw/pubsub: add ssl+sasl security to kafka 2019-11-25T09:18:00.142 INFO:teuthology.orchestra.run.smithi153:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2019-11-25T09:18:00.241 INFO:tasks.workunit.client.0.smithi153.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2019-11-25T09:18:00.245 INFO:tasks.workunit.client.0.smithi153.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-11-25T09:18:00.245 INFO:tasks.workunit.client.0.smithi153.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2019-11-25T09:18:00.802 INFO:tasks.workunit.client.0.smithi153.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2019-11-25T09:18:00.899 INFO:tasks.workunit.client.0.smithi153.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2019-11-25T09:18:00.949 INFO:tasks.workunit.client.0.smithi153.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2019-11-25T09:18:00.951 INFO:tasks.workunit.client.0.smithi153.stdout:make[1]: Entering directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-11-25T09:18:00.951 INFO:tasks.workunit.client.0.smithi153.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2019-11-25T09:18:00.984 INFO:tasks.workunit.client.0.smithi153.stdout:make[1]: Leaving directory `/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2019-11-25T09:18:01.037 DEBUG:teuthology.orchestra.remote:smithi153:/home/ubuntu/cephtest/workunits.list.client.0 is 4KB 2019-11-25T09:18:01.084 INFO:tasks.workunit:Running workunits matching cls/test_cls_lock.sh on client.0... 2019-11-25T09:18:01.084 INFO:tasks.workunit:Running workunit cls/test_cls_lock.sh... 2019-11-25T09:18:01.085 INFO:teuthology.orchestra.run.smithi153:workunit test cls/test_cls_lock.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=12b9659be24ae0e3d791f80d81504ab2770c1343 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_lock.sh 2019-11-25T09:18:01.142 INFO:tasks.workunit.client.0.smithi153.stdout:Running main() from gmock_main.cc 2019-11-25T09:18:01.142 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] Running 11 tests from 1 test suite. 2019-11-25T09:18:01.142 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment set-up. 2019-11-25T09:18:01.143 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 11 tests from ClsLock 2019-11-25T09:18:01.143 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestMultiLocking 2019-11-25T09:18:04.235 INFO:tasks.workunit.client.0.smithi153.stdout:lock: mylock 2019-11-25T09:18:04.235 INFO:tasks.workunit.client.0.smithi153.stdout: lock_type: exclusive 2019-11-25T09:18:04.236 INFO:tasks.workunit.client.0.smithi153.stdout: tag: 2019-11-25T09:18:04.236 INFO:tasks.workunit.client.0.smithi153.stdout: lockers: 2019-11-25T09:18:04.236 INFO:tasks.workunit.client.0.smithi153.stdout: client.4498 expiration=2019-11-25T09:20:04.102395+0000 addr=v1:172.21.15.153:0/1629981625 cookie= 2019-11-25T09:18:04.390 INFO:tasks.workunit.client.0.smithi153.stdout:lock: mylock 2019-11-25T09:18:04.391 INFO:tasks.workunit.client.0.smithi153.stdout: lock_type: shared 2019-11-25T09:18:04.391 INFO:tasks.workunit.client.0.smithi153.stdout: tag: 2019-11-25T09:18:04.391 INFO:tasks.workunit.client.0.smithi153.stdout: lockers: 2019-11-25T09:18:04.391 INFO:tasks.workunit.client.0.smithi153.stdout: client.4498 expiration=2019-11-25T09:20:04.338661+0000 addr=v1:172.21.15.153:0/1629981625 cookie= 2019-11-25T09:18:04.391 INFO:tasks.workunit.client.0.smithi153.stdout: client.4509 expiration=0.000000 addr=v1:172.21.15.153:0/2266348361 cookie= 2019-11-25T09:18:04.435 INFO:tasks.workunit.client.0.smithi153.stdout:lock: mylock 2019-11-25T09:18:04.435 INFO:tasks.workunit.client.0.smithi153.stdout: lock_type: shared 2019-11-25T09:18:04.436 INFO:tasks.workunit.client.0.smithi153.stdout: tag: 2019-11-25T09:18:04.436 INFO:tasks.workunit.client.0.smithi153.stdout: lockers: 2019-11-25T09:18:04.436 INFO:tasks.workunit.client.0.smithi153.stdout: client.4509 expiration=0.000000 addr=v1:172.21.15.153:0/2266348361 cookie= 2019-11-25T09:18:04.680 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestMultiLocking (3538 ms) 2019-11-25T09:18:04.680 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestMeta 2019-11-25T09:18:07.722 INFO:tasks.workunit.client.0.smithi153.stdout:lock: mylock 2019-11-25T09:18:07.722 INFO:tasks.workunit.client.0.smithi153.stdout: lock_type: shared 2019-11-25T09:18:07.723 INFO:tasks.workunit.client.0.smithi153.stdout: tag: 2019-11-25T09:18:07.723 INFO:tasks.workunit.client.0.smithi153.stdout: lockers: 2019-11-25T09:18:07.723 INFO:tasks.workunit.client.0.smithi153.stdout: client.4510 expiration=0.000000 addr=v1:172.21.15.153:0/1989658124 cookie= 2019-11-25T09:18:07.797 INFO:tasks.workunit.client.0.smithi153.stdout:lock: mylock 2019-11-25T09:18:07.798 INFO:tasks.workunit.client.0.smithi153.stdout: lock_type: exclusive 2019-11-25T09:18:07.798 INFO:tasks.workunit.client.0.smithi153.stdout: tag: new_tag 2019-11-25T09:18:07.798 INFO:tasks.workunit.client.0.smithi153.stdout: lockers: 2019-11-25T09:18:07.798 INFO:tasks.workunit.client.0.smithi153.stdout: client.4507 expiration=0.000000 addr=v1:172.21.15.153:0/2960627386 cookie= 2019-11-25T09:18:08.243 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestMeta (3562 ms) 2019-11-25T09:18:08.243 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestCookie 2019-11-25T09:18:11.435 INFO:tasks.workunit.client.0.smithi153.stdout:lock: mylock 2019-11-25T09:18:11.435 INFO:tasks.workunit.client.0.smithi153.stdout: lock_type: exclusive 2019-11-25T09:18:11.436 INFO:tasks.workunit.client.0.smithi153.stdout: tag: 2019-11-25T09:18:11.436 INFO:tasks.workunit.client.0.smithi153.stdout: lockers: 2019-11-25T09:18:11.504 INFO:tasks.workunit.client.0.smithi153.stdout:lock: mylock 2019-11-25T09:18:11.504 INFO:tasks.workunit.client.0.smithi153.stdout: lock_type: shared 2019-11-25T09:18:11.505 INFO:tasks.workunit.client.0.smithi153.stdout: tag: 2019-11-25T09:18:11.505 INFO:tasks.workunit.client.0.smithi153.stdout: lockers: 2019-11-25T09:18:11.505 INFO:tasks.workunit.client.0.smithi153.stdout: client.4516 expiration=0.000000 addr=v1:172.21.15.153:0/1858353685 cookie= 2019-11-25T09:18:11.505 INFO:tasks.workunit.client.0.smithi153.stdout: client.4516 expiration=0.000000 addr=v1:172.21.15.153:0/1858353685 cookie=new cookie 2019-11-25T09:18:11.671 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestCookie (3429 ms) 2019-11-25T09:18:11.671 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestMultipleLocks 2019-11-25T09:18:15.187 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestMultipleLocks (3516 ms) 2019-11-25T09:18:15.187 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestLockDuration 2019-11-25T09:18:23.492 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestLockDuration (8305 ms) 2019-11-25T09:18:23.493 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestAssertLocked 2019-11-25T09:18:26.704 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestAssertLocked (3211 ms) 2019-11-25T09:18:26.704 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestSetCookie 2019-11-25T09:18:30.078 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestSetCookie (3374 ms) 2019-11-25T09:18:30.078 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestRenew 2019-11-25T09:18:52.413 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestRenew (22332 ms) 2019-11-25T09:18:52.413 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralBasic 2019-11-25T09:18:59.877 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestExclusiveEphemeralBasic (7468 ms) 2019-11-25T09:18:59.878 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealEphemeral 2019-11-25T09:19:08.801 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealEphemeral (8924 ms) 2019-11-25T09:19:08.802 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] ClsLock.TestExclusiveEphemeralStealExclusive 2019-11-25T09:19:17.558 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] ClsLock.TestExclusiveEphemeralStealExclusive (8758 ms) 2019-11-25T09:19:17.559 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 11 tests from ClsLock (76418 ms total) 2019-11-25T09:19:17.559 INFO:tasks.workunit.client.0.smithi153.stdout: 2019-11-25T09:19:17.559 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment tear-down 2019-11-25T09:19:17.559 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] 11 tests from 1 test suite ran. (76418 ms total) 2019-11-25T09:19:17.559 INFO:tasks.workunit.client.0.smithi153.stdout:[ PASSED ] 11 tests. 2019-11-25T09:19:17.561 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-25T09:19:17.561 INFO:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-25T09:19:17.747 INFO:tasks.workunit:Running workunits matching cls/test_cls_log.sh on client.0... 2019-11-25T09:19:17.747 INFO:tasks.workunit:Running workunit cls/test_cls_log.sh... 2019-11-25T09:19:17.747 INFO:teuthology.orchestra.run.smithi153:workunit test cls/test_cls_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=12b9659be24ae0e3d791f80d81504ab2770c1343 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_log.sh 2019-11-25T09:19:17.835 INFO:tasks.workunit.client.0.smithi153.stdout:Running main() from gmock_main.cc 2019-11-25T09:19:17.835 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] Running 4 tests from 1 test suite. 2019-11-25T09:19:17.836 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment set-up. 2019-11-25T09:19:17.836 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 4 tests from cls_log 2019-11-25T09:19:17.836 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_log.test_log_add_same_time 2019-11-25T09:19:20.997 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_log.test_log_add_same_time (3162 ms) 2019-11-25T09:19:20.997 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_log.test_log_add_different_time 2019-11-25T09:19:25.526 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_log.test_log_add_different_time (4529 ms) 2019-11-25T09:19:25.526 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_log.trim_by_time 2019-11-25T09:19:30.482 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_log.trim_by_time (4956 ms) 2019-11-25T09:19:30.482 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_log.trim_by_marker 2019-11-25T09:19:35.027 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_log.trim_by_marker (4545 ms) 2019-11-25T09:19:35.028 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 4 tests from cls_log (17192 ms total) 2019-11-25T09:19:35.028 INFO:tasks.workunit.client.0.smithi153.stdout: 2019-11-25T09:19:35.028 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment tear-down 2019-11-25T09:19:35.028 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] 4 tests from 1 test suite ran. (17192 ms total) 2019-11-25T09:19:35.029 INFO:tasks.workunit.client.0.smithi153.stdout:[ PASSED ] 4 tests. 2019-11-25T09:19:35.030 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-25T09:19:35.030 INFO:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-25T09:19:35.134 INFO:tasks.workunit:Running workunits matching cls/test_cls_refcount.sh on client.0... 2019-11-25T09:19:35.135 INFO:tasks.workunit:Running workunit cls/test_cls_refcount.sh... 2019-11-25T09:19:35.135 INFO:teuthology.orchestra.run.smithi153:workunit test cls/test_cls_refcount.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=12b9659be24ae0e3d791f80d81504ab2770c1343 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_refcount.sh 2019-11-25T09:19:35.202 INFO:tasks.workunit.client.0.smithi153.stdout:Running main() from gmock_main.cc 2019-11-25T09:19:35.202 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] Running 5 tests from 1 test suite. 2019-11-25T09:19:35.202 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment set-up. 2019-11-25T09:19:35.202 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 5 tests from cls_rgw 2019-11-25T09:19:35.202 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.test_implicit 2019-11-25T09:19:38.679 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.test_implicit (3477 ms) 2019-11-25T09:19:38.679 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.test_implicit_idempotent 2019-11-25T09:19:42.189 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.test_implicit_idempotent (3510 ms) 2019-11-25T09:19:42.190 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.test_put_snap 2019-11-25T09:19:51.547 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T09:19:51.548+0000 3aa7b700 -1 osd.5 pg_epoch: 105 pg[25.6( v 104'4 (0'0,104'4] local-lis/les=101/102 n=2 ec=101/101 lis/c=101/101 les/c/f=102/102/0 sis=101) [5,1] r=0 lpr=101 crt=103'3 lcod 103'3 mlcod 103'3 active+clean+snaptrim trimq=[1~1]] removing snap head 2019-11-25T09:19:52.279 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.test_put_snap (10091 ms) 2019-11-25T09:19:52.280 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.test_explicit 2019-11-25T09:19:55.684 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.test_explicit (3405 ms) 2019-11-25T09:19:55.685 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.set 2019-11-25T09:19:59.201 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.set (3517 ms) 2019-11-25T09:19:59.202 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 5 tests from cls_rgw (24000 ms total) 2019-11-25T09:19:59.202 INFO:tasks.workunit.client.0.smithi153.stdout: 2019-11-25T09:19:59.202 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment tear-down 2019-11-25T09:19:59.202 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] 5 tests from 1 test suite ran. (24000 ms total) 2019-11-25T09:19:59.202 INFO:tasks.workunit.client.0.smithi153.stdout:[ PASSED ] 5 tests. 2019-11-25T09:19:59.204 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-25T09:19:59.204 INFO:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-25T09:19:59.275 INFO:tasks.workunit:Running workunits matching cls/test_cls_rgw.sh on client.0... 2019-11-25T09:19:59.275 INFO:tasks.workunit:Running workunit cls/test_cls_rgw.sh... 2019-11-25T09:19:59.276 INFO:teuthology.orchestra.run.smithi153:workunit test cls/test_cls_rgw.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=12b9659be24ae0e3d791f80d81504ab2770c1343 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_rgw.sh 2019-11-25T09:19:59.348 INFO:tasks.workunit.client.0.smithi153.stdout:Running main() from gmock_main.cc 2019-11-25T09:19:59.348 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] Running 13 tests from 1 test suite. 2019-11-25T09:19:59.349 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment set-up. 2019-11-25T09:19:59.349 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 13 tests from cls_rgw 2019-11-25T09:20:01.815 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.index_basic 2019-11-25T09:20:03.273 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.index_basic (1459 ms) 2019-11-25T09:20:03.273 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.index_multiple_obj_writers 2019-11-25T09:20:05.526 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.index_multiple_obj_writers (2253 ms) 2019-11-25T09:20:05.526 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.index_remove_object 2019-11-25T09:20:08.705 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.index_remove_object (3178 ms) 2019-11-25T09:20:08.705 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.index_suggest 2019-11-25T09:20:49.464 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.index_suggest (40760 ms) 2019-11-25T09:20:49.464 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.index_list 2019-11-25T09:20:50.664 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.index_list (1200 ms) 2019-11-25T09:20:50.664 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.bi_list 2019-11-25T09:20:54.777 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.bi_list (4114 ms) 2019-11-25T09:20:54.778 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.gc_set 2019-11-25T09:20:55.512 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.gc_set (734 ms) 2019-11-25T09:20:55.512 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.gc_list 2019-11-25T09:20:56.090 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.gc_list (578 ms) 2019-11-25T09:20:56.090 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.gc_defer 2019-11-25T09:21:03.668 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.gc_defer (7576 ms) 2019-11-25T09:21:03.669 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.usage_basic 2019-11-25T09:21:13.727 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.usage_basic (10061 ms) 2019-11-25T09:21:13.727 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.usage_clear_no_obj 2019-11-25T09:21:13.758 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.usage_clear_no_obj (30 ms) 2019-11-25T09:21:13.758 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.usage_clear 2019-11-25T09:21:21.040 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.usage_clear (7282 ms) 2019-11-25T09:21:21.040 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] cls_rgw.bi_log_trim 2019-11-25T09:21:24.024 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] cls_rgw.bi_log_trim (2985 ms) 2019-11-25T09:21:24.770 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 13 tests from cls_rgw (82211 ms total) 2019-11-25T09:21:24.770 INFO:tasks.workunit.client.0.smithi153.stdout: 2019-11-25T09:21:24.770 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment tear-down 2019-11-25T09:21:24.770 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] 13 tests from 1 test suite ran. (85423 ms total) 2019-11-25T09:21:24.770 INFO:tasks.workunit.client.0.smithi153.stdout:[ PASSED ] 13 tests. 2019-11-25T09:21:24.772 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-25T09:21:24.772 INFO:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-25T09:21:24.892 INFO:tasks.workunit:Running workunits matching rgw/test_rgw_gc_log.sh on client.0... 2019-11-25T09:21:24.892 INFO:tasks.workunit:Running workunit rgw/test_rgw_gc_log.sh... 2019-11-25T09:21:24.892 INFO:teuthology.orchestra.run.smithi153:workunit test rgw/test_rgw_gc_log.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=12b9659be24ae0e3d791f80d81504ab2770c1343 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rgw/test_rgw_gc_log.sh 2019-11-25T09:21:24.971 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] Running 6 tests from 1 test suite. 2019-11-25T09:21:24.972 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment set-up. 2019-11-25T09:21:27.169 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 6 tests from rgw_gc_log 2019-11-25T09:21:27.169 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] rgw_gc_log.init_existing_queue 2019-11-25T09:21:27.713 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] rgw_gc_log.init_existing_queue (543 ms) 2019-11-25T09:21:27.713 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] rgw_gc_log.init_existing_omap 2019-11-25T09:21:27.862 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] rgw_gc_log.init_existing_omap (150 ms) 2019-11-25T09:21:27.862 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] rgw_gc_log.enqueue1_after_init 2019-11-25T09:21:27.976 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] rgw_gc_log.enqueue1_after_init (114 ms) 2019-11-25T09:21:27.976 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] rgw_gc_log.enqueue2_before_init 2019-11-25T09:21:28.057 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] rgw_gc_log.enqueue2_before_init (80 ms) 2019-11-25T09:21:28.057 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] rgw_gc_log.defer1_after_init 2019-11-25T09:21:28.186 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] rgw_gc_log.defer1_after_init (130 ms) 2019-11-25T09:21:28.186 INFO:tasks.workunit.client.0.smithi153.stdout:[ RUN ] rgw_gc_log.defer2_before_init 2019-11-25T09:21:28.577 INFO:tasks.workunit.client.0.smithi153.stdout:[ OK ] rgw_gc_log.defer2_before_init (391 ms) 2019-11-25T09:21:28.578 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] 6 tests from rgw_gc_log (1408 ms total) 2019-11-25T09:21:28.578 INFO:tasks.workunit.client.0.smithi153.stdout: 2019-11-25T09:21:28.578 INFO:tasks.workunit.client.0.smithi153.stdout:[----------] Global test environment tear-down 2019-11-25T09:21:29.371 INFO:tasks.workunit.client.0.smithi153.stdout:[==========] 6 tests from 1 test suite ran. (4412 ms total) 2019-11-25T09:21:29.371 INFO:tasks.workunit.client.0.smithi153.stdout:[ PASSED ] 6 tests. 2019-11-25T09:21:29.374 INFO:teuthology.orchestra.run:Running command with timeout 3600 2019-11-25T09:21:29.374 INFO:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2019-11-25T09:21:29.460 INFO:tasks.workunit:Stopping ['cls/test_cls_lock.sh', 'cls/test_cls_log.sh', 'cls/test_cls_refcount.sh', 'cls/test_cls_rgw.sh', 'rgw/test_rgw_gc_log.sh'] on client.0... 2019-11-25T09:21:29.461 INFO:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2019-11-25T09:21:29.694 DEBUG:teuthology.parallel:result is None 2019-11-25T09:21:29.695 INFO:teuthology.orchestra.run.smithi153:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2019-11-25T09:21:29.752 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2019-11-25T09:21:29.753 INFO:teuthology.orchestra.run.smithi153:> rmdir -- /home/ubuntu/cephtest/mnt.0 2019-11-25T09:21:29.799 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2019-11-25T09:21:29.799 INFO:teuthology.run_tasks:Running task ragweed... 2019-11-25T09:21:29.857 DEBUG:tasks.ragweed:ragweed config is {'client.0': {'rgw_server': 'client.0', 'stages': 'prepare,check', 'default-branch': 'ceph-master'}} 2019-11-25T09:21:29.858 INFO:tasks.ragweed:Downloading ragweed... 2019-11-25T09:21:29.858 INFO:tasks.ragweed:Using branch 'wip-yuval-add-kafka-security2' for ragweed 2019-11-25T09:21:29.858 INFO:teuthology.orchestra.run.smithi153:> git clone -b wip-yuval-add-kafka-security2 git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-11-25T09:21:29.927 INFO:teuthology.orchestra.run.smithi153.stderr:fatal: Remote branch wip-yuval-add-kafka-security2 not found in upstream origin 2019-11-25T09:21:29.928 INFO:teuthology.orchestra.run.smithi153.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-11-25T09:21:29.929 DEBUG:teuthology.orchestra.run:got remote process result: 128 2019-11-25T09:21:29.929 INFO:teuthology.orchestra.run.smithi153:> git clone -b ceph-master git://git.ceph.com/git/ragweed.git /home/ubuntu/cephtest/ragweed 2019-11-25T09:21:30.024 INFO:teuthology.orchestra.run.smithi153.stdout:Cloning into '/home/ubuntu/cephtest/ragweed'... 2019-11-25T09:21:30.158 INFO:tasks.ragweed:Creating rgw users... 2019-11-25T09:21:30.158 DEBUG:tasks.ragweed:Creating user sysuser.client.0 on client.0 2019-11-25T09:21:30.159 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid sysuser.client.0 --display-name 'Mr. sysuser.client.0' --access-key KKYRZVPTXCBGAAGNSTRR --secret kHTttG6OKHX1bTSrU7/vFWIfuw6POXxAPD2zecvyHYtnwBe8Zu0u7Q== --email sysuser.client.0+test@test.test --system true 2019-11-25T09:21:30.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.321+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.346+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:30.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.346+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.348 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.351+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:30.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.351+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.357 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.355+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:30.357 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.355+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.363 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.367+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:30.493 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.497+0000 7f5a6a7a5ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:21:30.493 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.497+0000 7f5a6a7a5ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:21:30.493 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.497+0000 7f5a6a7a5ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:21:30.493 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.497+0000 7f5a6a7a5ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:21:30.494 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.497+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.507+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:30.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.507+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.514 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.518+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:21:30.514 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.518+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.525 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.529+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:30.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.529+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.540 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.543+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:21:30.541 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.543+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.549+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:30.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.549+0000 7f5a6a7a5ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:21:30.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.549+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.555 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.559+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:30.555 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.559+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.567+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:30.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.567+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.580+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:21:30.577 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.580+0000 7f5a6a7a5ec0 20 zonegroup default 2019-11-25T09:21:30.577 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.580+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.584 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.587+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:30.584 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.587+0000 7f5a6a7a5ec0 10 Cannot find current period zone using local zone 2019-11-25T09:21:30.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.587+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.595 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.598+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:30.596 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.598+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.612+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:30.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.612+0000 7f5a6a7a5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:30.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.629+0000 7f5a6a7a5ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:21:30.626 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.629+0000 7f5a6a7a5ec0 20 zone default found 2019-11-25T09:21:30.626 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.629+0000 7f5a6a7a5ec0 20 started sync module instance, tier type = 2019-11-25T09:21:30.627 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:30.629+0000 7f5a6a7a5ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:21:30.998 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.001+0000 7f5a6a7a5ec0 20 add_watcher() i=0 2019-11-25T09:21:30.998 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.001+0000 7f5a6a7a5ec0 20 add_watcher() i=1 2019-11-25T09:21:30.998 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.001+0000 7f5a6a7a5ec0 20 add_watcher() i=2 2019-11-25T09:21:30.999 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.001+0000 7f5a6a7a5ec0 20 add_watcher() i=3 2019-11-25T09:21:30.999 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.001+0000 7f5a6a7a5ec0 20 add_watcher() i=4 2019-11-25T09:21:30.999 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.001+0000 7f5a6a7a5ec0 20 add_watcher() i=5 2019-11-25T09:21:31.000 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.001+0000 7f5a6a7a5ec0 20 add_watcher() i=6 2019-11-25T09:21:31.025 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.029+0000 7f5a6a7a5ec0 20 add_watcher() i=7 2019-11-25T09:21:31.025 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.029+0000 7f5a6a7a5ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:21:31.025 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.029+0000 7f59abfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:21:31.038 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.042+0000 7f5a581d4700 0 -- >> v1:172.21.15.153:6789/0 conn(0x559e5938d190 legacy=0x559e593906a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T09:21:31.153 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:31.156+0000 7f5a581d4700 0 -- 172.21.15.153:0/2202734917 >> [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] conn(0x559e5939e2a0 msgr2=0x559e5939e6b0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:21:32.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:32.973+0000 7f59967fc700 20 reqs_thread_entry: start 2019-11-25T09:21:32.970 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:32.973+0000 7f598ffff700 20 reqs_thread_entry: start 2019-11-25T09:21:32.970 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:32.973+0000 7f5a6a7a5ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:21:32.971 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:32.974+0000 7f598e7fc700 20 reqs_thread_entry: start 2019-11-25T09:21:32.971 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:32.974+0000 7f5a6a7a5ec0 20 get_system_obj_state: rctx=0x7fff32d93b98 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x559e59408a80 s->prefetch_data=0 2019-11-25T09:21:32.971 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:32.974+0000 7f5a6a7a5ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-11-25T09:21:32.998 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.002+0000 7f5a6a7a5ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x0 2019-11-25T09:21:32.999 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.002+0000 7f5a6a7a5ec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-25T09:21:32.999 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.002+0000 7f5a6a7a5ec0 20 get_system_obj_state: rctx=0x7fff32d93ba8 obj=default.rgw.meta:users.email:sysuser.client.0+test@test.test state=0x559e5940a810 s->prefetch_data=0 2019-11-25T09:21:32.999 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.002+0000 7f5a6a7a5ec0 10 cache get: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test : miss 2019-11-25T09:21:33.011 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.015+0000 7f5a6a7a5ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x0 2019-11-25T09:21:33.011 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.015+0000 7f5a6a7a5ec0 10 adding default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-11-25T09:21:33.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.015+0000 7f5a6a7a5ec0 20 get_system_obj_state: rctx=0x7fff32d93ba8 obj=default.rgw.meta:users.keys:KKYRZVPTXCBGAAGNSTRR state=0x559e5940a810 s->prefetch_data=0 2019-11-25T09:21:33.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.015+0000 7f5a6a7a5ec0 10 cache get: name=default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR : miss 2019-11-25T09:21:33.025 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.029+0000 7f5a6a7a5ec0 10 cache put: name=default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR info.flags=0x0 2019-11-25T09:21:33.025 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.029+0000 7f5a6a7a5ec0 10 adding default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR to cache LRU end 2019-11-25T09:21:33.026 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.029+0000 7f5a6a7a5ec0 20 get_system_obj_state: rctx=0x7fff32d937b8 obj=default.rgw.meta:users.keys:KKYRZVPTXCBGAAGNSTRR state=0x559e5940a810 s->prefetch_data=0 2019-11-25T09:21:33.026 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.029+0000 7f5a6a7a5ec0 10 cache get: name=default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR : type miss (requested=0x6, cached=0x0) 2019-11-25T09:21:33.036 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.040+0000 7f5a6a7a5ec0 10 cache put: name=default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR info.flags=0x0 2019-11-25T09:21:33.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.040+0000 7f5a6a7a5ec0 10 moving default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR to cache LRU end 2019-11-25T09:21:33.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.040+0000 7f5a6a7a5ec0 20 get_system_obj_state: rctx=0x7fff32d93938 obj=default.rgw.meta:users.keys:KKYRZVPTXCBGAAGNSTRR state=0x559e5940a810 s->prefetch_data=0 2019-11-25T09:21:33.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.040+0000 7f5a6a7a5ec0 10 cache get: name=default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR : type miss (requested=0x6, cached=0x0) 2019-11-25T09:21:33.047 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.051+0000 7f5a6a7a5ec0 10 cache put: name=default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR info.flags=0x0 2019-11-25T09:21:33.047 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.051+0000 7f5a6a7a5ec0 10 moving default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR to cache LRU end 2019-11-25T09:21:33.105 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.107+0000 7f5a6a7a5ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-11-25T09:21:33.105 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.107+0000 7f5a6a7a5ec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-25T09:21:33.105 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.107+0000 7f5a6a7a5ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=637 2019-11-25T09:21:33.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.162+0000 7f5a4d7fa700 10 RGWWatcher::handle_notify() notify_id 536870912000 cookie 94138590002160 notifier 4615 bl.length()=637 2019-11-25T09:21:33.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.162+0000 7f5a4d7fa700 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x17 2019-11-25T09:21:33.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.162+0000 7f5a4d7fa700 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-25T09:21:33.525 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.530+0000 7f5a6a7a5ec0 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-11-25T09:21:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.530+0000 7f5a6a7a5ec0 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-11-25T09:21:33.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.530+0000 7f5a6a7a5ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=202 2019-11-25T09:21:33.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.551+0000 7f5a4d7fa700 10 RGWWatcher::handle_notify() notify_id 536870912001 cookie 94138590007264 notifier 4615 bl.length()=202 2019-11-25T09:21:33.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.551+0000 7f5a4d7fa700 10 cache put: name=default.rgw.meta+users.email+sysuser.client.0+test@test.test info.flags=0x7 2019-11-25T09:21:33.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:33.551+0000 7f5a4d7fa700 10 moving default.rgw.meta+users.email+sysuser.client.0+test@test.test to cache LRU end 2019-11-25T09:21:34.112 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.117+0000 7f5a6a7a5ec0 10 cache put: name=default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR info.flags=0x7 2019-11-25T09:21:34.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.117+0000 7f5a6a7a5ec0 10 moving default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR to cache LRU end 2019-11-25T09:21:34.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.117+0000 7f5a6a7a5ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=190 2019-11-25T09:21:34.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.132+0000 7f5a4d7fa700 10 RGWWatcher::handle_notify() notify_id 536870912002 cookie 94138590002160 notifier 4615 bl.length()=190 2019-11-25T09:21:34.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.132+0000 7f5a4d7fa700 10 cache put: name=default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR info.flags=0x7 2019-11-25T09:21:34.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.132+0000 7f5a4d7fa700 10 moving default.rgw.meta+users.keys+KKYRZVPTXCBGAAGNSTRR to cache LRU end 2019-11-25T09:21:34.377 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2019-11-25T09:21:34.377 INFO:teuthology.orchestra.run.smithi153.stdout: "user_id": "sysuser.client.0", 2019-11-25T09:21:34.378 INFO:teuthology.orchestra.run.smithi153.stdout: "display_name": "Mr. sysuser.client.0", 2019-11-25T09:21:34.378 INFO:teuthology.orchestra.run.smithi153.stdout: "email": "sysuser.client.0+test@test.test", 2019-11-25T09:21:34.378 INFO:teuthology.orchestra.run.smithi153.stdout: "suspended": 0, 2019-11-25T09:21:34.378 INFO:teuthology.orchestra.run.smithi153.stdout: "max_buckets": 1000, 2019-11-25T09:21:34.378 INFO:teuthology.orchestra.run.smithi153.stdout: "subusers": [], 2019-11-25T09:21:34.378 INFO:teuthology.orchestra.run.smithi153.stdout: "keys": [ 2019-11-25T09:21:34.378 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T09:21:34.379 INFO:teuthology.orchestra.run.smithi153.stdout: "user": "sysuser.client.0", 2019-11-25T09:21:34.379 INFO:teuthology.orchestra.run.smithi153.stdout: "access_key": "KKYRZVPTXCBGAAGNSTRR", 2019-11-25T09:21:34.379 INFO:teuthology.orchestra.run.smithi153.stdout: "secret_key": "kHTttG6OKHX1bTSrU7/vFWIfuw6POXxAPD2zecvyHYtnwBe8Zu0u7Q==" 2019-11-25T09:21:34.379 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T09:21:34.379 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T09:21:34.379 INFO:teuthology.orchestra.run.smithi153.stdout: "swift_keys": [], 2019-11-25T09:21:34.379 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": [], 2019-11-25T09:21:34.379 INFO:teuthology.orchestra.run.smithi153.stdout: "op_mask": "read, write, delete", 2019-11-25T09:21:34.380 INFO:teuthology.orchestra.run.smithi153.stdout: "system": "true", 2019-11-25T09:21:34.380 INFO:teuthology.orchestra.run.smithi153.stdout: "default_placement": "", 2019-11-25T09:21:34.380 INFO:teuthology.orchestra.run.smithi153.stdout: "default_storage_class": "", 2019-11-25T09:21:34.380 INFO:teuthology.orchestra.run.smithi153.stdout: "placement_tags": [], 2019-11-25T09:21:34.380 INFO:teuthology.orchestra.run.smithi153.stdout: "bucket_quota": { 2019-11-25T09:21:34.380 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T09:21:34.380 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T09:21:34.381 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T09:21:34.381 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T09:21:34.381 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T09:21:34.381 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T09:21:34.381 INFO:teuthology.orchestra.run.smithi153.stdout: "user_quota": { 2019-11-25T09:21:34.381 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T09:21:34.381 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T09:21:34.382 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T09:21:34.382 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T09:21:34.382 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T09:21:34.382 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T09:21:34.382 INFO:teuthology.orchestra.run.smithi153.stdout: "temp_url_keys": [], 2019-11-25T09:21:34.382 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "rgw", 2019-11-25T09:21:34.382 INFO:teuthology.orchestra.run.smithi153.stdout: "mfa_ids": [] 2019-11-25T09:21:34.382 INFO:teuthology.orchestra.run.smithi153.stdout:} 2019-11-25T09:21:34.383 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:34.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.433+0000 7f5a6a7a5ec0 20 remove_watcher() i=0 2019-11-25T09:21:34.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.433+0000 7f5a6a7a5ec0 2 removed watcher, disabling cache 2019-11-25T09:21:34.490 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.493+0000 7f5a6a7a5ec0 20 remove_watcher() i=1 2019-11-25T09:21:34.538 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.542+0000 7f5a6a7a5ec0 20 remove_watcher() i=2 2019-11-25T09:21:34.607 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.611+0000 7f5a6a7a5ec0 20 remove_watcher() i=3 2019-11-25T09:21:34.655 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.660+0000 7f5a6a7a5ec0 20 remove_watcher() i=4 2019-11-25T09:21:34.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.703+0000 7f5a6a7a5ec0 20 remove_watcher() i=5 2019-11-25T09:21:34.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.754+0000 7f5a6a7a5ec0 20 remove_watcher() i=6 2019-11-25T09:21:34.789 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.793+0000 7f5a6a7a5ec0 20 remove_watcher() i=7 2019-11-25T09:21:34.799 DEBUG:tasks.ragweed:Creating user ragweed.client.0 on client.0 2019-11-25T09:21:34.799 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid ragweed.client.0 --display-name 'Mr. ragweed.client.0' --access-key CZHRSUPRPHZWHQIHEBMH --secret LcKi1lvEN2qI07VOGZVLRQxzmQ6NBkETZ2Qxr6bRZZAL3VbkX1xtdw== --email ragweed.client.0+test@test.test --system false 2019-11-25T09:21:34.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.944+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:34.972 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.976+0000 7f134e8ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:34.972 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.976+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:34.981 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.985+0000 7f134e8ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:34.981 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.985+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:34.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.994+0000 7f134e8ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:34.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:34.994+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.017 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.021+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:35.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.231+0000 7f134e8ffec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:21:35.227 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.231+0000 7f134e8ffec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:21:35.227 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.231+0000 7f134e8ffec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:21:35.227 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.231+0000 7f134e8ffec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:21:35.227 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.231+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.243+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:35.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.243+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.251 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.256+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:21:35.252 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.256+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.268 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.272+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:35.268 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.272+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.291+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:21:35.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.291+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.305+0000 7f134e8ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:35.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.305+0000 7f134e8ffec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:21:35.301 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.305+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.310 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.314+0000 7f134e8ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:35.310 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.314+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.323 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.327+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:35.323 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.327+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.338 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.340+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:21:35.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.340+0000 7f134e8ffec0 20 zonegroup default 2019-11-25T09:21:35.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.340+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.348+0000 7f134e8ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:35.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.348+0000 7f134e8ffec0 10 Cannot find current period zone using local zone 2019-11-25T09:21:35.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.348+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.351 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.355+0000 7f134e8ffec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:21:35.351 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.355+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.368+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:21:35.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.368+0000 7f134e8ffec0 20 rados->read ofs=0 len=0 2019-11-25T09:21:35.376 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.381+0000 7f134e8ffec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:21:35.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.381+0000 7f134e8ffec0 20 zone default found 2019-11-25T09:21:35.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.381+0000 7f134e8ffec0 20 started sync module instance, tier type = 2019-11-25T09:21:35.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.381+0000 7f134e8ffec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:21:35.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.702+0000 7f134e8ffec0 20 add_watcher() i=0 2019-11-25T09:21:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.702+0000 7f134e8ffec0 20 add_watcher() i=1 2019-11-25T09:21:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.702+0000 7f134e8ffec0 20 add_watcher() i=2 2019-11-25T09:21:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.702+0000 7f134e8ffec0 20 add_watcher() i=3 2019-11-25T09:21:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.702+0000 7f134e8ffec0 20 add_watcher() i=4 2019-11-25T09:21:35.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.702+0000 7f134e8ffec0 20 add_watcher() i=5 2019-11-25T09:21:35.721 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.707+0000 7f134e8ffec0 20 add_watcher() i=6 2019-11-25T09:21:35.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.732+0000 7f134e8ffec0 20 add_watcher() i=7 2019-11-25T09:21:35.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.732+0000 7f134e8ffec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:21:35.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.732+0000 7f128ffff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:21:35.742 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.746+0000 7f133bb2d700 0 -- 172.21.15.153:0/3884123590 >> [v2:172.21.15.153:3300/0,v1:172.21.15.153:6789/0] conn(0x55be03297170 msgr2=0x55be03299590 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:21:35.767 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:35.770+0000 7f133bb2d700 0 -- 172.21.15.153:0/1461723557 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55be0329e620 msgr2=0x55be032a0a40 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:21:37.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.408+0000 7f127a7fc700 20 reqs_thread_entry: start 2019-11-25T09:21:37.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.408+0000 7f1273fff700 20 reqs_thread_entry: start 2019-11-25T09:21:37.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.408+0000 7f134e8ffec0 20 init_complete bucket index max shards: 0 2019-11-25T09:21:37.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.409+0000 7f12727fc700 20 reqs_thread_entry: start 2019-11-25T09:21:37.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.409+0000 7f134e8ffec0 20 get_system_obj_state: rctx=0x7ffca22b0e58 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x55be03314190 s->prefetch_data=0 2019-11-25T09:21:37.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.409+0000 7f134e8ffec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-11-25T09:21:37.417 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.421+0000 7f134e8ffec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x0 2019-11-25T09:21:37.417 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.421+0000 7f134e8ffec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-25T09:21:37.417 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.421+0000 7f134e8ffec0 20 get_system_obj_state: rctx=0x7ffca22b0e68 obj=default.rgw.meta:users.email:ragweed.client.0+test@test.test state=0x55be03315f20 s->prefetch_data=0 2019-11-25T09:21:37.417 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.421+0000 7f134e8ffec0 10 cache get: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test : miss 2019-11-25T09:21:37.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.431+0000 7f134e8ffec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x0 2019-11-25T09:21:37.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.431+0000 7f134e8ffec0 10 adding default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-11-25T09:21:37.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.431+0000 7f134e8ffec0 20 get_system_obj_state: rctx=0x7ffca22b0e68 obj=default.rgw.meta:users.keys:CZHRSUPRPHZWHQIHEBMH state=0x55be03315f20 s->prefetch_data=0 2019-11-25T09:21:37.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.431+0000 7f134e8ffec0 10 cache get: name=default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH : miss 2019-11-25T09:21:37.445 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.450+0000 7f134e8ffec0 10 cache put: name=default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH info.flags=0x0 2019-11-25T09:21:37.446 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.450+0000 7f134e8ffec0 10 adding default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH to cache LRU end 2019-11-25T09:21:37.446 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.450+0000 7f134e8ffec0 20 get_system_obj_state: rctx=0x7ffca22b0a78 obj=default.rgw.meta:users.keys:CZHRSUPRPHZWHQIHEBMH state=0x55be03315f20 s->prefetch_data=0 2019-11-25T09:21:37.446 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.450+0000 7f134e8ffec0 10 cache get: name=default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH : type miss (requested=0x6, cached=0x0) 2019-11-25T09:21:37.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.458+0000 7f134e8ffec0 10 cache put: name=default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH info.flags=0x0 2019-11-25T09:21:37.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.458+0000 7f134e8ffec0 10 moving default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH to cache LRU end 2019-11-25T09:21:37.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.458+0000 7f134e8ffec0 20 get_system_obj_state: rctx=0x7ffca22b0bf8 obj=default.rgw.meta:users.keys:CZHRSUPRPHZWHQIHEBMH state=0x55be03315f20 s->prefetch_data=0 2019-11-25T09:21:37.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.458+0000 7f134e8ffec0 10 cache get: name=default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH : type miss (requested=0x6, cached=0x0) 2019-11-25T09:21:37.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.466+0000 7f134e8ffec0 10 cache put: name=default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH info.flags=0x0 2019-11-25T09:21:37.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.466+0000 7f134e8ffec0 10 moving default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH to cache LRU end 2019-11-25T09:21:37.531 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.533+0000 7f134e8ffec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-11-25T09:21:37.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.533+0000 7f134e8ffec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-25T09:21:37.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.533+0000 7f134e8ffec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=637 2019-11-25T09:21:37.557 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.561+0000 7f13217fa700 10 RGWWatcher::handle_notify() notify_id 536870912003 cookie 94274585124512 notifier 4627 bl.length()=637 2019-11-25T09:21:37.557 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.561+0000 7f13217fa700 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x17 2019-11-25T09:21:37.557 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.561+0000 7f13217fa700 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-25T09:21:37.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.923+0000 7f134e8ffec0 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-11-25T09:21:37.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.923+0000 7f134e8ffec0 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-11-25T09:21:37.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.923+0000 7f134e8ffec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=202 2019-11-25T09:21:37.934 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.938+0000 7f13217fa700 10 RGWWatcher::handle_notify() notify_id 536870912004 cookie 94274585118976 notifier 4627 bl.length()=202 2019-11-25T09:21:37.934 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.938+0000 7f13217fa700 10 cache put: name=default.rgw.meta+users.email+ragweed.client.0+test@test.test info.flags=0x7 2019-11-25T09:21:37.935 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:37.938+0000 7f13217fa700 10 moving default.rgw.meta+users.email+ragweed.client.0+test@test.test to cache LRU end 2019-11-25T09:21:38.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:38.798+0000 7f134e8ffec0 10 cache put: name=default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH info.flags=0x7 2019-11-25T09:21:38.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:38.798+0000 7f134e8ffec0 10 moving default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH to cache LRU end 2019-11-25T09:21:38.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:38.798+0000 7f134e8ffec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=190 2019-11-25T09:21:38.832 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:38.836+0000 7f13217fa700 10 RGWWatcher::handle_notify() notify_id 536870912000 cookie 94274585118160 notifier 4627 bl.length()=190 2019-11-25T09:21:38.832 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:38.836+0000 7f13217fa700 10 cache put: name=default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH info.flags=0x7 2019-11-25T09:21:38.833 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:38.836+0000 7f13217fa700 10 moving default.rgw.meta+users.keys+CZHRSUPRPHZWHQIHEBMH to cache LRU end 2019-11-25T09:21:38.935 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2019-11-25T09:21:38.936 INFO:teuthology.orchestra.run.smithi153.stdout: "user_id": "ragweed.client.0", 2019-11-25T09:21:38.936 INFO:teuthology.orchestra.run.smithi153.stdout: "display_name": "Mr. ragweed.client.0", 2019-11-25T09:21:38.936 INFO:teuthology.orchestra.run.smithi153.stdout: "email": "ragweed.client.0+test@test.test", 2019-11-25T09:21:38.936 INFO:teuthology.orchestra.run.smithi153.stdout: "suspended": 0, 2019-11-25T09:21:38.936 INFO:teuthology.orchestra.run.smithi153.stdout: "max_buckets": 1000, 2019-11-25T09:21:38.936 INFO:teuthology.orchestra.run.smithi153.stdout: "subusers": [], 2019-11-25T09:21:38.936 INFO:teuthology.orchestra.run.smithi153.stdout: "keys": [ 2019-11-25T09:21:38.937 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T09:21:38.937 INFO:teuthology.orchestra.run.smithi153.stdout: "user": "ragweed.client.0", 2019-11-25T09:21:38.937 INFO:teuthology.orchestra.run.smithi153.stdout: "access_key": "CZHRSUPRPHZWHQIHEBMH", 2019-11-25T09:21:38.937 INFO:teuthology.orchestra.run.smithi153.stdout: "secret_key": "LcKi1lvEN2qI07VOGZVLRQxzmQ6NBkETZ2Qxr6bRZZAL3VbkX1xtdw==" 2019-11-25T09:21:38.937 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T09:21:38.937 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T09:21:38.937 INFO:teuthology.orchestra.run.smithi153.stdout: "swift_keys": [], 2019-11-25T09:21:38.938 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": [], 2019-11-25T09:21:38.938 INFO:teuthology.orchestra.run.smithi153.stdout: "op_mask": "read, write, delete", 2019-11-25T09:21:38.938 INFO:teuthology.orchestra.run.smithi153.stdout: "system": "true", 2019-11-25T09:21:38.938 INFO:teuthology.orchestra.run.smithi153.stdout: "default_placement": "", 2019-11-25T09:21:38.938 INFO:teuthology.orchestra.run.smithi153.stdout: "default_storage_class": "", 2019-11-25T09:21:38.938 INFO:teuthology.orchestra.run.smithi153.stdout: "placement_tags": [], 2019-11-25T09:21:38.938 INFO:teuthology.orchestra.run.smithi153.stdout: "bucket_quota": { 2019-11-25T09:21:38.938 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T09:21:38.939 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T09:21:38.939 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T09:21:38.939 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T09:21:38.940 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T09:21:38.940 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T09:21:38.940 INFO:teuthology.orchestra.run.smithi153.stdout: "user_quota": { 2019-11-25T09:21:38.940 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T09:21:38.940 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T09:21:38.940 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T09:21:38.940 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T09:21:38.941 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T09:21:38.941 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T09:21:38.941 INFO:teuthology.orchestra.run.smithi153.stdout: "temp_url_keys": [], 2019-11-25T09:21:38.941 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "rgw", 2019-11-25T09:21:38.941 INFO:teuthology.orchestra.run.smithi153.stdout: "mfa_ids": [] 2019-11-25T09:21:38.941 INFO:teuthology.orchestra.run.smithi153.stdout:} 2019-11-25T09:21:38.941 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:38.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:38.971+0000 7f134e8ffec0 20 remove_watcher() i=0 2019-11-25T09:21:38.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:38.971+0000 7f134e8ffec0 2 removed watcher, disabling cache 2019-11-25T09:21:39.016 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:39.017+0000 7f134e8ffec0 20 remove_watcher() i=1 2019-11-25T09:21:39.045 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:39.047+0000 7f134e8ffec0 20 remove_watcher() i=2 2019-11-25T09:21:39.090 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:39.095+0000 7f134e8ffec0 20 remove_watcher() i=3 2019-11-25T09:21:39.125 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:39.130+0000 7f134e8ffec0 20 remove_watcher() i=4 2019-11-25T09:21:39.155 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:39.159+0000 7f134e8ffec0 20 remove_watcher() i=5 2019-11-25T09:21:39.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:39.192+0000 7f134e8ffec0 20 remove_watcher() i=6 2019-11-25T09:21:39.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:21:39.228+0000 7f134e8ffec0 20 remove_watcher() i=7 2019-11-25T09:21:39.230 INFO:tasks.ragweed:Configuring ragweed... 2019-11-25T09:21:39.230 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest/ragweed && ./bootstrap 2019-11-25T09:21:45.383 INFO:teuthology.orchestra.run.smithi153.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-11-25T09:21:45.485 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T09:21:45.527 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T09:21:45.528 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T09:21:45.528 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T09:21:45.528 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T09:21:45.534 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T09:21:46.230 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T09:21:46.650 INFO:teuthology.orchestra.run.smithi153.stdout:Package python2-pip-8.1.2-10.el7.noarch already installed and latest version 2019-11-25T09:21:47.548 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T09:21:47.548 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:21:47.548 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libevent-devel.x86_64 0:2.0.21-4.el7 will be installed 2019-11-25T09:21:47.550 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 will be installed 2019-11-25T09:21:47.553 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: xz-devel for package: libxml2-devel-2.9.1-6.el7_2.3.x86_64 2019-11-25T09:21:47.586 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libxslt-devel.x86_64 0:1.1.28-5.el7 will be installed 2019-11-25T09:21:47.587 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libgcrypt-devel for package: libxslt-devel-1.1.28-5.el7.x86_64 2019-11-25T09:21:47.590 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package zlib-devel.x86_64 0:1.2.7-18.el7 will be installed 2019-11-25T09:21:47.591 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:21:47.591 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libgcrypt-devel.x86_64 0:1.5.3-14.el7 will be installed 2019-11-25T09:21:47.593 INFO:teuthology.orchestra.run.smithi153.stdout:--> Processing Dependency: libgpg-error-devel for package: libgcrypt-devel-1.5.3-14.el7.x86_64 2019-11-25T09:21:47.596 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package xz-devel.x86_64 0:5.2.2-1.el7 will be installed 2019-11-25T09:21:47.598 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T09:21:47.599 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libgpg-error-devel.x86_64 0:1.12-3.el7 will be installed 2019-11-25T09:21:47.981 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T09:21:48.181 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:48.181 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T09:21:48.183 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:48.183 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:21:48.183 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T09:21:48.183 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:21:48.183 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T09:21:48.183 INFO:teuthology.orchestra.run.smithi153.stdout: libevent-devel x86_64 2.0.21-4.el7 base 85 k 2019-11-25T09:21:48.184 INFO:teuthology.orchestra.run.smithi153.stdout: libxml2-devel x86_64 2.9.1-6.el7_2.3 base 1.0 M 2019-11-25T09:21:48.184 INFO:teuthology.orchestra.run.smithi153.stdout: libxslt-devel x86_64 1.1.28-5.el7 base 309 k 2019-11-25T09:21:48.184 INFO:teuthology.orchestra.run.smithi153.stdout: zlib-devel x86_64 1.2.7-18.el7 base 50 k 2019-11-25T09:21:48.184 INFO:teuthology.orchestra.run.smithi153.stdout:Installing for dependencies: 2019-11-25T09:21:48.184 INFO:teuthology.orchestra.run.smithi153.stdout: libgcrypt-devel x86_64 1.5.3-14.el7 base 129 k 2019-11-25T09:21:48.184 INFO:teuthology.orchestra.run.smithi153.stdout: libgpg-error-devel x86_64 1.12-3.el7 base 16 k 2019-11-25T09:21:48.185 INFO:teuthology.orchestra.run.smithi153.stdout: xz-devel x86_64 5.2.2-1.el7 base 46 k 2019-11-25T09:21:48.185 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:48.185 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T09:21:48.185 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T09:21:48.185 INFO:teuthology.orchestra.run.smithi153.stdout:Install 4 Packages (+3 Dependent packages) 2019-11-25T09:21:48.186 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:48.186 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 1.7 M 2019-11-25T09:21:48.186 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 12 M 2019-11-25T09:21:48.186 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T09:21:48.498 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------------------- 2019-11-25T09:21:48.499 INFO:teuthology.orchestra.run.smithi153.stdout:Total 5.3 MB/s | 1.7 MB 00:00 2019-11-25T09:21:48.508 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T09:21:48.528 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T09:21:48.587 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T09:21:48.587 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T09:21:49.174 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgpg-error-devel-1.12-3.el7.x86_64 1/7 2019-11-25T09:21:49.500 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libgcrypt-devel-1.5.3-14.el7.x86_64 2/7 2019-11-25T09:21:49.783 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-11-25T09:21:50.482 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : xz-devel-5.2.2-1.el7.x86_64 4/7 2019-11-25T09:21:50.929 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libxml2-devel-2.9.1-6.el7_2.3.x86_64 5/7 2019-11-25T09:21:51.263 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libxslt-devel-1.1.28-5.el7.x86_64 6/7 2019-11-25T09:21:51.525 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libevent-devel-2.0.21-4.el7.x86_64 7/7 2019-11-25T09:21:51.633 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : xz-devel-5.2.2-1.el7.x86_64 1/7 2019-11-25T09:21:51.813 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libxml2-devel-2.9.1-6.el7_2.3.x86_64 2/7 2019-11-25T09:21:51.942 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : zlib-devel-1.2.7-18.el7.x86_64 3/7 2019-11-25T09:21:52.084 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libxslt-devel-1.1.28-5.el7.x86_64 4/7 2019-11-25T09:21:52.182 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgpg-error-devel-1.12-3.el7.x86_64 5/7 2019-11-25T09:21:52.285 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libevent-devel-2.0.21-4.el7.x86_64 6/7 2019-11-25T09:21:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libgcrypt-devel-1.5.3-14.el7.x86_64 7/7 2019-11-25T09:21:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:52.801 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T09:21:52.801 INFO:teuthology.orchestra.run.smithi153.stdout: libevent-devel.x86_64 0:2.0.21-4.el7 libxml2-devel.x86_64 0:2.9.1-6.el7_2.3 2019-11-25T09:21:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: libxslt-devel.x86_64 0:1.1.28-5.el7 zlib-devel.x86_64 0:1.2.7-18.el7 2019-11-25T09:21:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:52.802 INFO:teuthology.orchestra.run.smithi153.stdout:Dependency Installed: 2019-11-25T09:21:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: libgcrypt-devel.x86_64 0:1.5.3-14.el7 libgpg-error-devel.x86_64 0:1.12-3.el7 2019-11-25T09:21:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: xz-devel.x86_64 0:5.2.2-1.el7 2019-11-25T09:21:52.802 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:21:52.802 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T09:21:53.032 INFO:teuthology.orchestra.run.smithi153.stdout:New python executable in /home/ubuntu/cephtest/ragweed/virtualenv/bin/python 2019-11-25T09:21:54.509 INFO:teuthology.orchestra.run.smithi153.stdout:Installing setuptools, pip, wheel...done. 2019-11-25T09:21:55.050 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting pip 2019-11-25T09:21:55.191 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl (1.4MB) 2019-11-25T09:21:55.439 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: pip 2019-11-25T09:21:55.439 INFO:teuthology.orchestra.run.smithi153.stdout: Found existing installation: pip 9.0.1 2019-11-25T09:21:55.539 INFO:teuthology.orchestra.run.smithi153.stdout: Uninstalling pip-9.0.1: 2019-11-25T09:21:55.615 INFO:teuthology.orchestra.run.smithi153.stdout: Successfully uninstalled pip-9.0.1 2019-11-25T09:21:56.481 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed pip-19.3.1 2019-11-25T09:21:57.292 INFO:teuthology.orchestra.run.smithi153.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-25T09:21:57.350 INFO:teuthology.orchestra.run.smithi153.stdout:Requirement already satisfied: six in /usr/lib/python2.7/site-packages (1.9.0) 2019-11-25T09:21:58.163 INFO:teuthology.orchestra.run.smithi153.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-25T09:21:58.380 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting nose 2019-11-25T09:21:58.421 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl (154kB) 2019-11-25T09:21:58.597 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: nose 2019-11-25T09:21:58.672 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed nose-1.3.7 2019-11-25T09:21:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-25T09:21:59.781 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting setuptools==32.3.1 2019-11-25T09:21:59.827 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl (479kB) 2019-11-25T09:22:00.015 INFO:teuthology.orchestra.run.smithi153.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-11-25T09:22:00.016 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: setuptools 2019-11-25T09:22:00.016 INFO:teuthology.orchestra.run.smithi153.stdout: Found existing installation: setuptools 28.8.0 2019-11-25T09:22:00.035 INFO:teuthology.orchestra.run.smithi153.stdout: Uninstalling setuptools-28.8.0: 2019-11-25T09:22:00.045 INFO:teuthology.orchestra.run.smithi153.stdout: Successfully uninstalled setuptools-28.8.0 2019-11-25T09:22:00.191 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed setuptools-32.3.1 2019-11-25T09:22:00.707 INFO:teuthology.orchestra.run.smithi153.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-25T09:22:01.009 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting PyYAML 2019-11-25T09:22:01.061 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/e3/e8/b3212641ee2718d556df0f23f78de8303f068fe29cdaa7a91018849582fe/PyYAML-5.1.2.tar.gz (265kB) 2019-11-25T09:22:01.743 INFO:teuthology.orchestra.run.smithi153.stdout:Requirement already up-to-date: nose>=1.0.0 in ./virtualenv/lib/python2.7/site-packages (from -r requirements.txt (line 2)) (1.3.7) 2019-11-25T09:22:01.880 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting boto!=2.46.0,>=2.6.0 2019-11-25T09:22:01.891 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl (1.4MB) 2019-11-25T09:22:02.221 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting bunch>=1.0.0 2019-11-25T09:22:02.237 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/ef/bf/a4cf1779a4ffb4f610903fa08e15d1f4a8a2f4e3353a02afbe097c5bf4a8/bunch-1.0.1.tar.gz 2019-11-25T09:22:02.894 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting gevent>=1.0 2019-11-25T09:22:02.903 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl (5.0MB) 2019-11-25T09:22:03.410 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting httplib2 2019-11-25T09:22:03.419 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/ce/2e/87461bfbb7e561203b759b3f7f639e2144226604372830d00a8279960ae1/httplib2-0.14.0.tar.gz (219kB) 2019-11-25T09:22:04.541 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting lxml 2019-11-25T09:22:04.553 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/e4/f4/65d145cd6917131826050b0479be35aaccba2847b7f80fc4afc6bec6616b/lxml-4.4.1-cp27-cp27mu-manylinux1_x86_64.whl (5.7MB) 2019-11-25T09:22:05.077 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-11-25T09:22:05.087 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-25T09:22:05.161 INFO:teuthology.orchestra.run.smithi153.stdout:Building wheels for collected packages: PyYAML, bunch, httplib2 2019-11-25T09:22:05.162 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for PyYAML (setup.py): started 2019-11-25T09:22:05.827 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for PyYAML (setup.py): finished with status 'done' 2019-11-25T09:22:05.827 INFO:teuthology.orchestra.run.smithi153.stdout: Created wheel for PyYAML: filename=PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl size=46204 sha256=a2f6d55ce37f45862656f9432ac7e0f8371a8503595f03c9bffaa647205928fe 2019-11-25T09:22:05.828 INFO:teuthology.orchestra.run.smithi153.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030 2019-11-25T09:22:05.864 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for bunch (setup.py): started 2019-11-25T09:22:06.208 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for bunch (setup.py): finished with status 'done' 2019-11-25T09:22:06.208 INFO:teuthology.orchestra.run.smithi153.stdout: Created wheel for bunch: filename=bunch-1.0.1-cp27-none-any.whl size=8347 sha256=b3c9dddbfca4897b30e502adda75c21d9cef0799fdcda80ee6e63defcc3f0908 2019-11-25T09:22:06.208 INFO:teuthology.orchestra.run.smithi153.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c 2019-11-25T09:22:06.209 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for httplib2 (setup.py): started 2019-11-25T09:22:06.522 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for httplib2 (setup.py): finished with status 'done' 2019-11-25T09:22:06.522 INFO:teuthology.orchestra.run.smithi153.stdout: Created wheel for httplib2: filename=httplib2-0.14.0-cp27-none-any.whl size=96742 sha256=aefd81c7674090e728160c7f4e86f19454919743fe3a69c30b90908362ff05e4 2019-11-25T09:22:06.522 INFO:teuthology.orchestra.run.smithi153.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488 2019-11-25T09:22:06.527 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully built PyYAML bunch httplib2 2019-11-25T09:22:06.604 INFO:teuthology.orchestra.run.smithi153.stderr:ERROR: kubernetes 10.0.1 has requirement ipaddress>=1.0.17, but you'll have ipaddress 1.0.16 which is incompatible. 2019-11-25T09:22:06.604 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: PyYAML, boto, bunch, greenlet, gevent, httplib2, lxml 2019-11-25T09:22:06.605 INFO:teuthology.orchestra.run.smithi153.stdout: Found existing installation: PyYAML 3.10 2019-11-25T09:22:06.605 INFO:teuthology.orchestra.run.smithi153.stdout: Not uninstalling pyyaml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-11-25T09:22:06.605 INFO:teuthology.orchestra.run.smithi153.stdout: Can't uninstall 'PyYAML'. No files were found to uninstall. 2019-11-25T09:22:07.761 INFO:teuthology.orchestra.run.smithi153.stdout: Found existing installation: lxml 3.2.1 2019-11-25T09:22:07.763 INFO:teuthology.orchestra.run.smithi153.stdout: Not uninstalling lxml at /usr/lib64/python2.7/site-packages, outside environment /home/ubuntu/cephtest/ragweed/virtualenv 2019-11-25T09:22:07.763 INFO:teuthology.orchestra.run.smithi153.stdout: Can't uninstall 'lxml'. No files were found to uninstall. 2019-11-25T09:22:07.855 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed PyYAML-5.1.2 boto-2.49.0 bunch-1.0.1 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 lxml-4.4.1 2019-11-25T09:22:08.274 INFO:teuthology.orchestra.run.smithi153.stdout:running develop 2019-11-25T09:22:08.339 INFO:teuthology.orchestra.run.smithi153.stdout:running egg_info 2019-11-25T09:22:08.339 INFO:teuthology.orchestra.run.smithi153.stdout:creating ragweed.egg-info 2019-11-25T09:22:08.345 INFO:teuthology.orchestra.run.smithi153.stdout:writing requirements to ragweed.egg-info/requires.txt 2019-11-25T09:22:08.345 INFO:teuthology.orchestra.run.smithi153.stdout:writing ragweed.egg-info/PKG-INFO 2019-11-25T09:22:08.346 INFO:teuthology.orchestra.run.smithi153.stdout:writing top-level names to ragweed.egg-info/top_level.txt 2019-11-25T09:22:08.346 INFO:teuthology.orchestra.run.smithi153.stdout:writing dependency_links to ragweed.egg-info/dependency_links.txt 2019-11-25T09:22:08.346 INFO:teuthology.orchestra.run.smithi153.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-11-25T09:22:08.346 INFO:teuthology.orchestra.run.smithi153.stdout:reading manifest file 'ragweed.egg-info/SOURCES.txt' 2019-11-25T09:22:08.346 INFO:teuthology.orchestra.run.smithi153.stdout:writing manifest file 'ragweed.egg-info/SOURCES.txt' 2019-11-25T09:22:08.354 INFO:teuthology.orchestra.run.smithi153.stdout:running build_ext 2019-11-25T09:22:08.354 INFO:teuthology.orchestra.run.smithi153.stdout:Creating /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/ragweed.egg-link (link to .) 2019-11-25T09:22:08.354 INFO:teuthology.orchestra.run.smithi153.stdout:Adding ragweed 0.0.1 to easy-install.pth file 2019-11-25T09:22:08.354 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:22:08.354 INFO:teuthology.orchestra.run.smithi153.stdout:Installed /home/ubuntu/cephtest/ragweed 2019-11-25T09:22:08.354 INFO:teuthology.orchestra.run.smithi153.stdout:Processing dependencies for ragweed==0.0.1 2019-11-25T09:22:08.354 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for isodate>=0.4.4 2019-11-25T09:22:08.355 INFO:teuthology.orchestra.run.smithi153.stdout:Reading https://pypi.python.org/simple/isodate/ 2019-11-25T09:22:08.440 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading https://files.pythonhosted.org/packages/b1/80/fb8c13a4cd38eb5021dc3741a9e588e4d1de88d895c1910c6fc8a08b7a70/isodate-0.6.0.tar.gz#sha256=2e364a3d5759479cdb2d37cce6b9376ea504db2ff90252a2e5b7cc89cc9ff2d8 2019-11-25T09:22:08.489 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: isodate 0.6.0 2019-11-25T09:22:08.489 INFO:teuthology.orchestra.run.smithi153.stdout:Processing isodate-0.6.0.tar.gz 2019-11-25T09:22:08.506 INFO:teuthology.orchestra.run.smithi153.stdout:Writing /tmp/easy_install-Zsxvxj/isodate-0.6.0/setup.cfg 2019-11-25T09:22:08.506 INFO:teuthology.orchestra.run.smithi153.stdout:Running isodate-0.6.0/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Zsxvxj/isodate-0.6.0/egg-dist-tmp-_H9vT5 2019-11-25T09:22:08.815 INFO:teuthology.orchestra.run.smithi153.stderr:zip_safe flag not set; analyzing archive contents... 2019-11-25T09:22:08.830 INFO:teuthology.orchestra.run.smithi153.stdout:Moving isodate-0.6.0-py2.7.egg to /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-25T09:22:08.831 INFO:teuthology.orchestra.run.smithi153.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-11-25T09:22:08.832 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:22:08.833 INFO:teuthology.orchestra.run.smithi153.stdout:Installed /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages/isodate-0.6.0-py2.7.egg 2019-11-25T09:22:08.856 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for gevent==1.4.0 2019-11-25T09:22:08.856 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: gevent 1.4.0 2019-11-25T09:22:08.857 INFO:teuthology.orchestra.run.smithi153.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-11-25T09:22:08.858 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:22:08.859 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-25T09:22:08.859 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for bunch==1.0.1 2019-11-25T09:22:08.859 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: bunch 1.0.1 2019-11-25T09:22:08.859 INFO:teuthology.orchestra.run.smithi153.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-11-25T09:22:08.859 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:22:08.860 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-25T09:22:08.860 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for PyYAML==5.1.2 2019-11-25T09:22:08.860 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: PyYAML 5.1.2 2019-11-25T09:22:08.860 INFO:teuthology.orchestra.run.smithi153.stdout:Adding PyYAML 5.1.2 to easy-install.pth file 2019-11-25T09:22:08.861 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:22:08.861 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-25T09:22:08.862 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for boto==2.49.0 2019-11-25T09:22:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: boto 2.49.0 2019-11-25T09:22:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-11-25T09:22:08.863 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:22:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-25T09:22:08.863 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for six==1.9.0 2019-11-25T09:22:08.864 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: six 1.9.0 2019-11-25T09:22:08.864 INFO:teuthology.orchestra.run.smithi153.stdout:Adding six 1.9.0 to easy-install.pth file 2019-11-25T09:22:08.864 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:22:08.864 INFO:teuthology.orchestra.run.smithi153.stdout:Using /usr/lib/python2.7/site-packages 2019-11-25T09:22:08.870 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for greenlet==0.4.15 2019-11-25T09:22:08.870 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: greenlet 0.4.15 2019-11-25T09:22:08.870 INFO:teuthology.orchestra.run.smithi153.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-11-25T09:22:08.870 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:22:08.870 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/ragweed/virtualenv/lib/python2.7/site-packages 2019-11-25T09:22:08.871 INFO:teuthology.orchestra.run.smithi153.stdout:Finished processing dependencies for ragweed==0.0.1 2019-11-25T09:22:08.885 INFO:teuthology.orchestra.run.smithi153:> cat > /home/ubuntu/cephtest/archive/ragweed.client.0.conf 2019-11-25T09:22:08.970 INFO:tasks.ragweed:Configuring boto... 2019-11-25T09:22:08.971 INFO:teuthology.orchestra.run.smithi153:> cat > /home/ubuntu/cephtest/boto.cfg 2019-11-25T09:22:09.022 INFO:teuthology.orchestra.run.smithi153:ragweed tests against rgw> RAGWEED_CONF=/home/ubuntu/cephtest/archive/ragweed.client.0.conf RAGWEED_STAGES=prepare,check BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg /home/ubuntu/cephtest/ragweed/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/ragweed -v -a '!fails_on_rgw' 2019-11-25T09:22:47.378 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_multipart_defer_complete.test ... ok 2019-11-25T09:23:12.904 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_multipart_defer_update_complete.test ... ok 2019-11-25T09:23:37.104 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_multipart_index_versioning.test ... ok 2019-11-25T09:23:59.808 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_multipart_simple.test ... ok 2019-11-25T09:24:02.422 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:09:07.014 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:02.523 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:09:07.139 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:02.542 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:09:07.136 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:02.544 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:09:07.161 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:02.597 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:09:07.213 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:02.626 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:09:07.243 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:03.261 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:09:07.837 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:03.278 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:09:07.890 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:03.327 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:09:07.940 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:24:28.218 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_obj_storage_class.test ... ok 2019-11-25T09:25:36.512 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_obj_storage_class_copy.test ... ok 2019-11-25T09:26:05.512 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:11:10.107 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:05.515 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:11:10.111 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:05.529 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:11:10.140 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:05.529 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:11:10.146 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:05.561 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:11:10.178 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:05.563 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:11:10.182 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:05.648 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:11:10.266 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:05.671 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:11:10.267 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:05.676 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:11:10.295 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:26:37.080 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart.test ... ok 2019-11-25T09:28:17.749 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_obj_storage_class_multipart_copy.test ... ok 2019-11-25T09:28:50.509 INFO:teuthology.orchestra.run.smithi153.stderr:ragweed.tests.tests.r_test_small_obj_data.test ... ok 2019-11-25T09:28:50.509 INFO:teuthology.orchestra.run.smithi153.stderr: 2019-11-25T09:28:50.510 INFO:teuthology.orchestra.run.smithi153.stderr:---------------------------------------------------------------------- 2019-11-25T09:28:50.510 INFO:teuthology.orchestra.run.smithi153.stderr:Ran 9 tests in 401.047s 2019-11-25T09:28:50.510 INFO:teuthology.orchestra.run.smithi153.stderr: 2019-11-25T09:28:50.510 INFO:teuthology.orchestra.run.smithi153.stderr:OK 2019-11-25T09:28:50.540 INFO:tasks.ragweed:Cleaning up boto... 2019-11-25T09:28:50.540 INFO:teuthology.orchestra.run.smithi153:> rm /home/ubuntu/cephtest/boto.cfg 2019-11-25T09:28:50.603 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid sysuser.client.0 --purge-data 2019-11-25T09:28:50.718 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:50.729+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:50.752 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:50.764+0000 7f03b1670ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:50.753 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:50.764+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:50.763 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:50.774+0000 7f03b1670ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:50.763 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:50.774+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:50.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:50.784+0000 7f03b1670ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:50.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:50.784+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:50.795 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:50.807+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:51.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.023+0000 7f03b1670ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:28:51.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.023+0000 7f03b1670ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:28:51.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.023+0000 7f03b1670ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:28:51.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.023+0000 7f03b1670ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:28:51.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.023+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.024 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.036+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:51.024 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.036+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.040 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.052+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:28:51.041 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.052+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.051 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.063+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:51.052 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.063+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.072+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:28:51.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.072+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.065 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.077+0000 7f03b1670ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:51.065 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.077+0000 7f03b1670ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:28:51.065 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.077+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.069 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.081+0000 7f03b1670ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:51.069 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.081+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.077 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.089+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:51.077 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.089+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.100+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:28:51.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.100+0000 7f03b1670ec0 20 zonegroup default 2019-11-25T09:28:51.089 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.100+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.110 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.122+0000 7f03b1670ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:51.110 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.122+0000 7f03b1670ec0 10 Cannot find current period zone using local zone 2019-11-25T09:28:51.111 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.122+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.128+0000 7f03b1670ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:51.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.128+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.125 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.137+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:51.126 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.137+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:51.136 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.147+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:28:51.136 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.147+0000 7f03b1670ec0 20 zone default found 2019-11-25T09:28:51.136 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.147+0000 7f03b1670ec0 20 started sync module instance, tier type = 2019-11-25T09:28:51.136 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.147+0000 7f03b1670ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:28:51.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.439+0000 7f03b1670ec0 20 add_watcher() i=0 2019-11-25T09:28:51.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.439+0000 7f03b1670ec0 20 add_watcher() i=1 2019-11-25T09:28:51.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.439+0000 7f03b1670ec0 20 add_watcher() i=2 2019-11-25T09:28:51.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.439+0000 7f03b1670ec0 20 add_watcher() i=3 2019-11-25T09:28:51.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.439+0000 7f03b1670ec0 20 add_watcher() i=4 2019-11-25T09:28:51.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.439+0000 7f03b1670ec0 20 add_watcher() i=5 2019-11-25T09:28:51.434 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.446+0000 7f03b1670ec0 20 add_watcher() i=6 2019-11-25T09:28:51.473 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.484+0000 7f03b1670ec0 20 add_watcher() i=7 2019-11-25T09:28:51.473 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.484+0000 7f03b1670ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:28:51.473 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.484+0000 7f036f7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:28:51.486 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.499+0000 7f039e09d700 0 -- >> v1:172.21.15.153:6789/0 conn(0x5636b04ae000 legacy=0x5636b04b04d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T09:28:51.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:51.628+0000 7f039f09f700 0 -- 172.21.15.153:0/3027697306 >> [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] conn(0x5636b04bd2d0 msgr2=0x5636b04bf6f0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:28:53.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.444+0000 7f02ddffb700 20 reqs_thread_entry: start 2019-11-25T09:28:53.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.444+0000 7f02d77fe700 20 reqs_thread_entry: start 2019-11-25T09:28:53.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.444+0000 7f03b1670ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:28:53.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.444+0000 7f02d5ffb700 20 reqs_thread_entry: start 2019-11-25T09:28:53.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.445+0000 7f03b1670ec0 20 get_system_obj_state: rctx=0x7ffcac6dbf78 obj=default.rgw.meta:users.uid:sysuser.client.0 state=0x5636b0520d40 s->prefetch_data=0 2019-11-25T09:28:53.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.445+0000 7f03b1670ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : miss 2019-11-25T09:28:53.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.465+0000 7f03b1670ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x16 2019-11-25T09:28:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.465+0000 7f03b1670ec0 10 adding default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-25T09:28:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.465+0000 7f03b1670ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:28:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.465+0000 7f03b1670ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:28:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.465+0000 7f03b1670ec0 10 cache get: name=default.rgw.meta+users.uid+sysuser.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-25T09:28:53.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.465+0000 7f03b1670ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:53.471 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.484+0000 7f03b1670ec0 20 rados_obj.operate() r=0 bl.length=448 2019-11-25T09:28:53.472 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.484+0000 7f03b1670ec0 10 cache put: name=default.rgw.meta+users.uid+sysuser.client.0 info.flags=0x11 2019-11-25T09:28:53.472 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.484+0000 7f03b1670ec0 10 moving default.rgw.meta+users.uid+sysuser.client.0 to cache LRU end 2019-11-25T09:28:53.477 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.490+0000 7f03b1670ec0 10 removing key index: KKYRZVPTXCBGAAGNSTRR 2019-11-25T09:28:53.478 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.490+0000 7f03b1670ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=170 2019-11-25T09:28:53.486 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.499+0000 7f037ffff700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 94792885809664 notifier 4490 bl.length()=170 2019-11-25T09:28:53.633 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.646+0000 7f03b1670ec0 10 removing email index: sysuser.client.0+test@test.test 2019-11-25T09:28:53.633 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.646+0000 7f03b1670ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=182 2019-11-25T09:28:53.643 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.656+0000 7f037ffff700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94792885808032 notifier 4490 bl.length()=182 2019-11-25T09:28:53.939 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.952+0000 7f03b1670ec0 10 removing user buckets index 2019-11-25T09:28:53.939 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.952+0000 7f03b1670ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=173 2019-11-25T09:28:53.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:53.967+0000 7f037ffff700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 94792885811552 notifier 4490 bl.length()=173 2019-11-25T09:28:54.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.259+0000 7f03b1670ec0 10 removing user index: sysuser.client.0 2019-11-25T09:28:54.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.259+0000 7f03b1670ec0 10 removing default.rgw.meta+users.uid+sysuser.client.0 from cache 2019-11-25T09:28:54.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.259+0000 7f03b1670ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=165 2019-11-25T09:28:54.261 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.274+0000 7f037ffff700 10 RGWWatcher::handle_notify() notify_id 579820584967 cookie 94792885809664 notifier 4490 bl.length()=165 2019-11-25T09:28:54.447 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.459+0000 7f03b1670ec0 20 remove_watcher() i=0 2019-11-25T09:28:54.447 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.459+0000 7f03b1670ec0 2 removed watcher, disabling cache 2019-11-25T09:28:54.490 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.503+0000 7f03b1670ec0 20 remove_watcher() i=1 2019-11-25T09:28:54.534 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.547+0000 7f03b1670ec0 20 remove_watcher() i=2 2019-11-25T09:28:54.574 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.587+0000 7f03b1670ec0 20 remove_watcher() i=3 2019-11-25T09:28:54.606 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.618+0000 7f03b1670ec0 20 remove_watcher() i=4 2019-11-25T09:28:54.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.653+0000 7f03b1670ec0 20 remove_watcher() i=5 2019-11-25T09:28:54.669 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.681+0000 7f03b1670ec0 20 remove_watcher() i=6 2019-11-25T09:28:54.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.712+0000 7f03b1670ec0 20 remove_watcher() i=7 2019-11-25T09:28:54.706 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid ragweed.client.0 --purge-data 2019-11-25T09:28:54.817 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.830+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:54.837 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.849+0000 7f1386e35ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:54.837 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.849+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:54.841 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.854+0000 7f1386e35ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:54.841 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.854+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:54.846 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.859+0000 7f1386e35ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:54.846 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.859+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:54.866 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:54.872+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:55.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.066+0000 7f1386e35ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:28:55.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.066+0000 7f1386e35ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:28:55.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.066+0000 7f1386e35ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:28:55.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.066+0000 7f1386e35ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:28:55.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.066+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.073+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:55.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.073+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.073 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.085+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:28:55.073 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.085+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.080 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.093+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:55.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.093+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.101+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:28:55.095 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.101+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.100 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.112+0000 7f1386e35ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:55.100 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.112+0000 7f1386e35ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:28:55.100 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.112+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.108 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.121+0000 7f1386e35ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:55.109 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.121+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.123 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.136+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:55.124 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.136+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.150+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:28:55.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.150+0000 7f1386e35ec0 20 zonegroup default 2019-11-25T09:28:55.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.150+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.163+0000 7f1386e35ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:55.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.163+0000 7f1386e35ec0 10 Cannot find current period zone using local zone 2019-11-25T09:28:55.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.163+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.159 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.172+0000 7f1386e35ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:28:55.160 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.172+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.182+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:28:55.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.182+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:55.185 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.198+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:28:55.185 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.198+0000 7f1386e35ec0 20 zone default found 2019-11-25T09:28:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.198+0000 7f1386e35ec0 20 started sync module instance, tier type = 2019-11-25T09:28:55.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.198+0000 7f1386e35ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:28:55.578 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.590+0000 7f1386e35ec0 20 add_watcher() i=0 2019-11-25T09:28:55.578 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.590+0000 7f1386e35ec0 20 add_watcher() i=1 2019-11-25T09:28:55.578 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.590+0000 7f1386e35ec0 20 add_watcher() i=2 2019-11-25T09:28:55.578 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.590+0000 7f1386e35ec0 20 add_watcher() i=3 2019-11-25T09:28:55.578 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.590+0000 7f1386e35ec0 20 add_watcher() i=4 2019-11-25T09:28:55.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.590+0000 7f1386e35ec0 20 add_watcher() i=5 2019-11-25T09:28:55.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.590+0000 7f1386e35ec0 20 add_watcher() i=6 2019-11-25T09:28:55.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.615+0000 7f1386e35ec0 20 add_watcher() i=7 2019-11-25T09:28:55.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.615+0000 7f1386e35ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:28:55.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.615+0000 7f1344fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:28:55.604 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.617+0000 7f136bfff700 0 -- 172.21.15.153:0/2059665274 >> v1:172.21.15.81:6789/0 conn(0x556bbc7c8280 legacy=0x556bbc7ca790 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:28:55.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:55.654+0000 7f136bfff700 0 -- 172.21.15.153:0/1375702857 >> [v2:172.21.15.81:6800/9718,v1:172.21.15.81:6801/9718] conn(0x556bbc7d80f0 msgr2=0x556bbc7da510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:28:57.245 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.257+0000 7f12b6ffd700 20 reqs_thread_entry: start 2019-11-25T09:28:57.245 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.257+0000 7f12b4ff9700 20 reqs_thread_entry: start 2019-11-25T09:28:57.245 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.258+0000 7f1386e35ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:28:57.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.258+0000 7f12aaffd700 20 reqs_thread_entry: start 2019-11-25T09:28:57.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.258+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c217b8 obj=default.rgw.meta:users.uid:ragweed.client.0 state=0x556bbc841960 s->prefetch_data=0 2019-11-25T09:28:57.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.258+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : miss 2019-11-25T09:28:57.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.311+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x16 2019-11-25T09:28:57.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.311+0000 7f1386e35ec0 10 adding default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-25T09:28:57.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.311+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:28:57.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.311+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:28:57.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.311+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+users.uid+ragweed.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-25T09:28:57.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.311+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:57.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.331+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=448 2019-11-25T09:28:57.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.331+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+users.uid+ragweed.client.0 info.flags=0x11 2019-11-25T09:28:57.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.331+0000 7f1386e35ec0 10 moving default.rgw.meta+users.uid+ragweed.client.0 to cache LRU end 2019-11-25T09:28:57.352 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.364+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-conf state=0x556bbc843280 s->prefetch_data=0 2019-11-25T09:28:57.352 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.364+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : miss 2019-11-25T09:28:57.370 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.382+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x6 2019-11-25T09:28:57.370 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.382+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-11-25T09:28:57.370 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.382+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:28:57.370 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.382+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:28:57.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.382+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x1, cached=0x6) 2019-11-25T09:28:57.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.382+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:57.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.390+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=200 2019-11-25T09:28:57.378 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.390+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x1 2019-11-25T09:28:57.378 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.390+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-11-25T09:28:57.378 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.390+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 state=0x556bbc845840 s->prefetch_data=0 2019-11-25T09:28:57.378 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.390+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 : miss 2019-11-25T09:28:57.390 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 info.flags=0x16 2019-11-25T09:28:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 to cache LRU end 2019-11-25T09:28:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:28:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:28:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:28:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:28:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:28:57.391 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:28:57.392 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.403+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:28:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.417+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=292 2019-11-25T09:28:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.417+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 info.flags=0x13 2019-11-25T09:28:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.417+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 to cache LRU end 2019-11-25T09:28:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.417+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:28:57.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.417+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:28:57.406 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.417+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:57.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.443+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-conf[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1]) start [] num_entries 1100 2019-11-25T09:28:57.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_complete[] 2019-11-25T09:28:57.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_defer_update_complete[] 2019-11-25T09:28:57.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_index_versioning[] 2019-11-25T09:28:57.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_multipart_simple[] 2019-11-25T09:28:57.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class[] 2019-11-25T09:28:57.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_copy[] 2019-11-25T09:28:57.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart[] 2019-11-25T09:28:57.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_obj_storage_class_multipart_copy[] 2019-11-25T09:28:57.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got tests/r_test_small_obj_data[] 2019-11-25T09:28:57.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.492+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:57.490 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.503+0000 7f1386e35ec0 10 manifest: total_size = 5843 2019-11-25T09:28:57.490 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.503+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.13 2019-11-25T09:28:57.491 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.503+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_multipart_defer_complete state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:57.491 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.503+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:57.491 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.503+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:57.664 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.676+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:57.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.695+0000 7f1386e35ec0 10 manifest: total_size = 3913 2019-11-25T09:28:57.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.695+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.30 2019-11-25T09:28:57.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.695+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_multipart_defer_update_complete state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:57.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.695+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:57.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.695+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:57.974 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.986+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:57.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.996+0000 7f1386e35ec0 10 manifest: total_size = 5688 2019-11-25T09:28:57.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.996+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.56 2019-11-25T09:28:57.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.996+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_multipart_index_versioning state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:57.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.996+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:57.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:57.996+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:58.269 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.282+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:58.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.298+0000 7f1386e35ec0 10 manifest: total_size = 3795 2019-11-25T09:28:58.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.298+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.72 2019-11-25T09:28:58.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.298+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_multipart_simple state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:58.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.298+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:58.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.299+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:58.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.544+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:58.543 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.556+0000 7f1386e35ec0 10 manifest: total_size = 5893 2019-11-25T09:28:58.543 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.556+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.87 2019-11-25T09:28:58.544 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.556+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_obj_storage_class state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:58.544 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.556+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:58.544 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.556+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:58.831 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.844+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:58.850 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.862+0000 7f1386e35ec0 10 manifest: total_size = 6053 2019-11-25T09:28:58.850 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.862+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.120 2019-11-25T09:28:58.850 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.862+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_obj_storage_class_copy state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:58.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.862+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:58.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:58.863+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:59.050 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.062+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:59.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.072+0000 7f1386e35ec0 10 manifest: total_size = 6135 2019-11-25T09:28:59.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.072+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.189 2019-11-25T09:28:59.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.072+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:59.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.072+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:59.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.072+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:59.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.254+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:59.261 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.274+0000 7f1386e35ec0 10 manifest: total_size = 3992 2019-11-25T09:28:59.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.274+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.246 2019-11-25T09:28:59.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.274+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_obj_storage_class_multipart_copy state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:59.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.274+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:59.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.274+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:59.510 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.522+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:59.520 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.533+0000 7f1386e35ec0 10 manifest: total_size = 29155 2019-11-25T09:28:59.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.533+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.322 2019-11-25T09:28:59.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.533+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-conf:tests/r_test_small_obj_data state=0x556bbc84aa68 s->prefetch_data=0 2019-11-25T09:28:59.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.533+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aa68 2019-11-25T09:28:59.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.533+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 2019-11-25T09:28:59.840 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.853+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-conf[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1]) start _multipart_[] num_entries 1001 2019-11-25T09:28:59.841 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.853+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:28:59.959 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:28:59.972+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-conf[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:00.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.066+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-conf state=0x556bbc843280 s->prefetch_data=0 2019-11-25T09:29:00.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.066+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:00.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.082+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-conf info.flags=0x16 2019-11-25T09:29:00.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.082+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-conf to cache LRU end 2019-11-25T09:29:00.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.082+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:00.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.082+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:00.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.082+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-conf : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:00.071 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.082+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-conf from cache 2019-11-25T09:29:00.071 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.082+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=162 2019-11-25T09:29:00.102 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.114+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584960 cookie 93921212041840 notifier 4499 bl.length()=162 2019-11-25T09:29:00.211 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.224+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-conf:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.1 from cache 2019-11-25T09:29:00.211 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.224+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=219 2019-11-25T09:29:00.228 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.241+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 93921212045040 notifier 4499 bl.length()=219 2019-11-25T09:29:00.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.788+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x556bbc84a780 s->prefetch_data=0 2019-11-25T09:29:00.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.788+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : miss 2019-11-25T09:29:00.792 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.805+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x6 2019-11-25T09:29:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.805+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-11-25T09:29:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.805+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.805+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.805+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:00.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.805+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:00.801 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.814+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=229 2019-11-25T09:29:00.801 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.814+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x1 2019-11-25T09:29:00.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.814+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-11-25T09:29:00.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.814+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 state=0x556bbc843280 s->prefetch_data=0 2019-11-25T09:29:00.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.814+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 : miss 2019-11-25T09:29:00.833 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 info.flags=0x16 2019-11-25T09:29:00.833 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 to cache LRU end 2019-11-25T09:29:00.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:00.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:00.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:00.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:00.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:00.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:00.835 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.845+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:00.865 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.878+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=321 2019-11-25T09:29:00.866 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.878+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 info.flags=0x13 2019-11-25T09:29:00.866 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.878+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 to cache LRU end 2019-11-25T09:29:00.866 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.878+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:00.866 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.878+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:00.866 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.878+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 2019-11-25T09:29:00.915 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.927+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-complete.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2]) start [] num_entries 1100 2019-11-25T09:29:00.971 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.984+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-11-25T09:29:00.972 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:00.984+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x556bbc84a9d8 s->prefetch_data=0 2019-11-25T09:29:00.991 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.003+0000 7f1386e35ec0 10 manifest: total_size = 18874368 2019-11-25T09:29:00.991 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.003+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.20 2019-11-25T09:29:00.991 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.003+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-defer-complete.1:foo state=0x556bbc84a9d8 s->prefetch_data=0 2019-11-25T09:29:00.991 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.003+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84a9d8 2019-11-25T09:29:00.991 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.003+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 2019-11-25T09:29:01.147 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:01.148 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:01.148 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:01.148 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:01.148 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:01.148 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:01.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:01.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:01.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:01.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:01.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:01.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=5242880 2019-11-25T09:29:01.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:01.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2019-11-25T09:29:01.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=14680064 stripe_ofs=14680064 part_ofs=10485760 rule->part_size=5242880 2019-11-25T09:29:01.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:01.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=18874368 part_ofs=10485760 rule->part_size=5242880 2019-11-25T09:29:01.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=15728640 stripe_ofs=15728640 part_ofs=15728640 rule->part_size=3145728 2019-11-25T09:29:01.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=3145728 rules.size()=2 2019-11-25T09:29:01.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=19922944 part_ofs=15728640 rule->part_size=3145728 2019-11-25T09:29:01.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=18874368 stripe_ofs=18874368 part_ofs=18874368 rule->part_size=3145728 2019-11-25T09:29:01.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.160+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.2tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.20 2019-11-25T09:29:01.256 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.269+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-complete.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2]) start _multipart_[] num_entries 1001 2019-11-25T09:29:01.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.269+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:01.279 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.292+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-complete.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:01.406 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.419+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-complete.1 state=0x556bbc84a780 s->prefetch_data=0 2019-11-25T09:29:01.407 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.419+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:01.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.437+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 info.flags=0x16 2019-11-25T09:29:01.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.437+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 to cache LRU end 2019-11-25T09:29:01.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.437+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:01.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.437+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:01.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.437+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:01.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.437+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-complete.1 from cache 2019-11-25T09:29:01.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.437+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=191 2019-11-25T09:29:01.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.460+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 93921212045040 notifier 4499 bl.length()=191 2019-11-25T09:29:01.718 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.731+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2 from cache 2019-11-25T09:29:01.718 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.731+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=248 2019-11-25T09:29:01.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:01.752+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584968 cookie 93921212052048 notifier 4499 bl.length()=248 2019-11-25T09:29:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.686+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x556bbc843280 s->prefetch_data=0 2019-11-25T09:29:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.686+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : miss 2019-11-25T09:29:02.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.708+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x6 2019-11-25T09:29:02.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.708+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-11-25T09:29:02.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.708+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:02.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.708+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:02.696 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.708+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:02.696 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.708+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:02.712 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.725+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=236 2019-11-25T09:29:02.712 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.725+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x1 2019-11-25T09:29:02.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.725+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-11-25T09:29:02.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.725+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 state=0x556bbc848c20 s->prefetch_data=0 2019-11-25T09:29:02.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.725+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 : miss 2019-11-25T09:29:02.732 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 info.flags=0x16 2019-11-25T09:29:02.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 to cache LRU end 2019-11-25T09:29:02.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:02.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:02.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:02.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:02.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:02.734 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.745+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:02.758 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.770+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=328 2019-11-25T09:29:02.758 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.770+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 info.flags=0x13 2019-11-25T09:29:02.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.770+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 to cache LRU end 2019-11-25T09:29:02.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.770+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:02.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.770+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:02.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.770+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 2019-11-25T09:29:02.817 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.830+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-defer-update-complete.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3]) start [] num_entries 1100 2019-11-25T09:29:02.893 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.905+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-11-25T09:29:02.893 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.905+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x556bbc84a9d8 s->prefetch_data=0 2019-11-25T09:29:02.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.918+0000 7f1386e35ec0 10 manifest: total_size = 74198 2019-11-25T09:29:02.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.918+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.40 2019-11-25T09:29:02.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.918+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-defer-update-complete.1:foo state=0x556bbc84a9d8 s->prefetch_data=0 2019-11-25T09:29:02.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.918+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84a9d8 2019-11-25T09:29:02.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:02.918+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 2019-11-25T09:29:03.086 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-25T09:29:03.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-11-25T09:29:03.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-11-25T09:29:03.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-25T09:29:03.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-11-25T09:29:03.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2019-11-25T09:29:03.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-25T09:29:03.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=20613 2019-11-25T09:29:03.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2019-11-25T09:29:03.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2019-11-25T09:29:03.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4256143 part_ofs=61839 rule->part_size=12359 2019-11-25T09:29:03.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2019-11-25T09:29:03.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.098+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.40 2019-11-25T09:29:03.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.207+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-defer-update-complete.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3]) start _multipart_[] num_entries 1001 2019-11-25T09:29:03.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.207+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:03.217 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.230+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-defer-update-complete.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.331+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-multipart-defer-update-complete.1 state=0x556bbc848c20 s->prefetch_data=0 2019-11-25T09:29:03.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.331+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:03.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.345+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 info.flags=0x16 2019-11-25T09:29:03.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.345+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 to cache LRU end 2019-11-25T09:29:03.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.345+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:03.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.345+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:03.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.345+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:03.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.345+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-defer-update-complete.1 from cache 2019-11-25T09:29:03.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.345+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=198 2019-11-25T09:29:03.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.355+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584969 cookie 93921212046944 notifier 4499 bl.length()=198 2019-11-25T09:29:03.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.828+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-defer-update-complete.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3 from cache 2019-11-25T09:29:03.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.828+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=255 2019-11-25T09:29:03.830 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:03.843+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 93921212048640 notifier 4499 bl.length()=255 2019-11-25T09:29:04.133 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.146+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x556bbc848c20 s->prefetch_data=0 2019-11-25T09:29:04.134 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.146+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : miss 2019-11-25T09:29:04.146 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.159+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x6 2019-11-25T09:29:04.146 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.159+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-11-25T09:29:04.146 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.159+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:04.147 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.159+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:04.147 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.159+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:04.147 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.159+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:04.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.170+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=231 2019-11-25T09:29:04.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.170+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x1 2019-11-25T09:29:04.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.170+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-11-25T09:29:04.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.170+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 state=0x556bbc843280 s->prefetch_data=0 2019-11-25T09:29:04.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.170+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 : miss 2019-11-25T09:29:04.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 info.flags=0x16 2019-11-25T09:29:04.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 to cache LRU end 2019-11-25T09:29:04.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:04.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:04.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:04.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:04.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:04.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:04.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.186+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:04.205 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.218+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=323 2019-11-25T09:29:04.205 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.218+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 info.flags=0x13 2019-11-25T09:29:04.205 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.218+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 to cache LRU end 2019-11-25T09:29:04.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.218+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:04.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.218+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:04.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.218+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:04.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.255+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-index-versioning.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4]) start [] num_entries 1100 2019-11-25T09:29:04.331 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.344+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo[Vcout5rMCpGSJZ5uT.G9RP53khR165j] 2019-11-25T09:29:04.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.344+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:04.361 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.374+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-index-versioning.1:foo state=0x556bbc84ada8 s->prefetch_data=0 2019-11-25T09:29:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.384+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to yj4vn4yuv3k4kad2r3uo2y1tqvdtpxxc 2019-11-25T09:29:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.384+0000 7f1386e35ec0 20 get_obj_state_impl: setting s->olh_tag to veggokd9lb8vy5k16ggihj0wj2jb5kf1 2019-11-25T09:29:04.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.384+0000 7f1386e35ec0 20 bucket_index_guard_olh_op(): olh_state.olh_tag=veggokd9lb8vy5k16ggihj0wj2jb5kf1 2019-11-25T09:29:04.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.467+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:04.551 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.564+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:04.551 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.564+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:04.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.598+0000 7f1386e35ec0 20 olh_log_entry: op=2 key=[] 2019-11-25T09:29:04.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.598+0000 7f1386e35ec0 20 olh_log_entry: op=3 key=foo[Vcout5rMCpGSJZ5uT.G9RP53khR165j] 2019-11-25T09:29:04.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.598+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-index-versioning.1:_:Vcout5rMCpGSJZ5uT.G9RP53khR165j_foo state=0x556bbc8516f8 s->prefetch_data=0 2019-11-25T09:29:04.599 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.612+0000 7f1386e35ec0 10 manifest: total_size = 74198 2019-11-25T09:29:04.599 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.612+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.55 2019-11-25T09:29:04.599 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.612+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-index-versioning.1:_:Vcout5rMCpGSJZ5uT.G9RP53khR165j_foo state=0x556bbc8516f8 s->prefetch_data=0 2019-11-25T09:29:04.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.612+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8516f8 2019-11-25T09:29:04.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.612+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:04.767 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-25T09:29:04.768 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=20613 2019-11-25T09:29:04.768 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=20613 stripe_ofs=20613 part_ofs=20613 rule->part_size=20613 2019-11-25T09:29:04.768 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-25T09:29:04.768 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4214917 part_ofs=20613 rule->part_size=20613 2019-11-25T09:29:04.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=41226 stripe_ofs=41226 part_ofs=41226 rule->part_size=20613 2019-11-25T09:29:04.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=20613 rules.size()=2 2019-11-25T09:29:04.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4235530 part_ofs=41226 rule->part_size=20613 2019-11-25T09:29:04.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=61839 stripe_ofs=61839 part_ofs=61839 rule->part_size=12359 2019-11-25T09:29:04.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=12359 rules.size()=2 2019-11-25T09:29:04.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4256143 part_ofs=61839 rule->part_size=12359 2019-11-25T09:29:04.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=74198 stripe_ofs=74198 part_ofs=74198 rule->part_size=12359 2019-11-25T09:29:04.770 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.780+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.55 2019-11-25T09:29:04.924 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.936+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:04.924 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:04.936+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:05.045 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.058+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:05.108 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.121+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 2019-11-25T09:29:05.109 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.121+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-index-versioning.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4]) start _multipart_[] num_entries 1001 2019-11-25T09:29:05.109 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.121+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:05.130 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.143+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-index-versioning.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:05.223 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.235+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-multipart-index-versioning.1 state=0x556bbc856c70 s->prefetch_data=0 2019-11-25T09:29:05.223 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.236+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.250+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 info.flags=0x16 2019-11-25T09:29:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.250+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 to cache LRU end 2019-11-25T09:29:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.250+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:05.238 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.250+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.250+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.250+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-index-versioning.1 from cache 2019-11-25T09:29:05.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.250+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=193 2019-11-25T09:29:05.256 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.269+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584970 cookie 93921212046944 notifier 4499 bl.length()=193 2019-11-25T09:29:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.626+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-index-versioning.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4 from cache 2019-11-25T09:29:05.614 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.626+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=250 2019-11-25T09:29:05.628 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:05.641+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584963 cookie 93921212049840 notifier 4499 bl.length()=250 2019-11-25T09:29:06.457 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.470+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x556bbc848c20 s->prefetch_data=0 2019-11-25T09:29:06.457 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.470+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : miss 2019-11-25T09:29:06.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.479+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x6 2019-11-25T09:29:06.467 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.479+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-11-25T09:29:06.467 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.479+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:06.467 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.479+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:06.467 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.479+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:06.468 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.479+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:06.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.492+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=221 2019-11-25T09:29:06.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.492+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x1 2019-11-25T09:29:06.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.492+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-11-25T09:29:06.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.492+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 state=0x556bbc843280 s->prefetch_data=0 2019-11-25T09:29:06.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.492+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 : miss 2019-11-25T09:29:06.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 info.flags=0x16 2019-11-25T09:29:06.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 to cache LRU end 2019-11-25T09:29:06.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:06.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:06.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:06.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:06.496 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:06.496 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:06.496 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.508+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:06.514 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.527+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=313 2019-11-25T09:29:06.514 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.527+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 info.flags=0x13 2019-11-25T09:29:06.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.527+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 to cache LRU end 2019-11-25T09:29:06.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.527+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:06.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.527+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:06.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.527+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 2019-11-25T09:29:06.536 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.549+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-multipart-simple.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5]) start [] num_entries 1100 2019-11-25T09:29:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.577+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo[] 2019-11-25T09:29:06.565 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.577+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x556bbc84aca8 s->prefetch_data=0 2019-11-25T09:29:06.581 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.594+0000 7f1386e35ec0 10 manifest: total_size = 1848 2019-11-25T09:29:06.582 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.594+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.71 2019-11-25T09:29:06.582 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.594+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-multipart-simple.1:foo state=0x556bbc84aca8 s->prefetch_data=0 2019-11-25T09:29:06.582 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.594+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc84aca8 2019-11-25T09:29:06.582 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.594+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 2019-11-25T09:29:06.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-25T09:29:06.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-11-25T09:29:06.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-11-25T09:29:06.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-25T09:29:06.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-11-25T09:29:06.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=515 2019-11-25T09:29:06.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-25T09:29:06.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=515 2019-11-25T09:29:06.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1545 stripe_ofs=1545 part_ofs=1545 rule->part_size=303 2019-11-25T09:29:06.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=303 rules.size()=2 2019-11-25T09:29:06.751 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195849 part_ofs=1545 rule->part_size=303 2019-11-25T09:29:06.751 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1848 stripe_ofs=1848 part_ofs=1848 rule->part_size=303 2019-11-25T09:29:06.751 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.761+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.28tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.71 2019-11-25T09:29:06.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.865+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-multipart-simple.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5]) start _multipart_[] num_entries 1001 2019-11-25T09:29:06.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.865+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:06.874 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.887+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-multipart-simple.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:06.981 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.994+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-multipart-simple.1 state=0x556bbc843280 s->prefetch_data=0 2019-11-25T09:29:06.981 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:06.994+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:06.991 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.004+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 info.flags=0x16 2019-11-25T09:29:06.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.004+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 to cache LRU end 2019-11-25T09:29:06.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.004+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:06.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.004+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:06.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.004+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:06.993 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.004+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-multipart-simple.1 from cache 2019-11-25T09:29:06.993 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.004+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=183 2019-11-25T09:29:07.000 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.013+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584971 cookie 93921212052048 notifier 4499 bl.length()=183 2019-11-25T09:29:07.084 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.097+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-multipart-simple.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5 from cache 2019-11-25T09:29:07.084 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.097+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=240 2019-11-25T09:29:07.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.126+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 93921212058176 notifier 4499 bl.length()=240 2019-11-25T09:29:07.297 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.309+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x556bbc848c20 s->prefetch_data=0 2019-11-25T09:29:07.297 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.309+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : miss 2019-11-25T09:29:07.306 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.319+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x6 2019-11-25T09:29:07.306 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.319+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-11-25T09:29:07.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.319+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:07.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.319+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:07.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.319+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:07.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.319+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:07.314 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.327+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=227 2019-11-25T09:29:07.314 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.327+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x1 2019-11-25T09:29:07.314 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.327+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-11-25T09:29:07.315 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.327+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 state=0x556bbc795bf0 s->prefetch_data=0 2019-11-25T09:29:07.315 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.327+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 : miss 2019-11-25T09:29:07.327 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 info.flags=0x16 2019-11-25T09:29:07.327 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 to cache LRU end 2019-11-25T09:29:07.328 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:07.328 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:07.328 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:07.329 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:07.329 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:07.329 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:07.331 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.338+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.352+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=319 2019-11-25T09:29:07.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.352+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 info.flags=0x13 2019-11-25T09:29:07.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.352+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 to cache LRU end 2019-11-25T09:29:07.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.352+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:07.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.352+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:07.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.352+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:07.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.371+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-copy.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7]) start [] num_entries 1100 2019-11-25T09:29:07.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-11-25T09:29:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-11-25T09:29:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-11-25T09:29:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-11-25T09:29:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-11-25T09:29:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-11-25T09:29:07.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-11-25T09:29:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-11-25T09:29:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-11-25T09:29:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-11-25T09:29:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-11-25T09:29:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-11-25T09:29:07.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.442+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:07.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.463+0000 7f1386e35ec0 10 manifest: total_size = 5242880 2019-11-25T09:29:07.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.463+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.108 2019-11-25T09:29:07.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.463+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:07.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.463+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:07.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.463+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:07.565 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.578+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:07.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.578+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:07.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.578+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:07.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.578+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-25T09:29:07.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.578+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.8tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.108 2019-11-25T09:29:07.615 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.628+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:07.624 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.637+0000 7f1386e35ec0 10 manifest: total_size = 5242880 2019-11-25T09:29:07.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.637+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.109 2019-11-25T09:29:07.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.637+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-FROZEN state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:07.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.637+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:07.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.637+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:07.753 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.766+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:07.753 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.766+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:07.753 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.766+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:07.753 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.766+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-25T09:29:07.754 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.766+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.24tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.109 2019-11-25T09:29:07.789 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.802+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:07.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.815+0000 7f1386e35ec0 10 manifest: total_size = 5242880 2019-11-25T09:29:07.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.815+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _U9K2jFSjDc7h8eIerIrYpfHDoqAK14x 2019-11-25T09:29:07.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.815+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-LUKEWARM state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:07.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.815+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:07.803 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.815+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:07.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.927+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:07.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.927+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:07.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.927+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:07.915 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.927+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-25T09:29:07.915 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.927+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: _U9K2jFSjDc7h8eIerIrYpfHDoqAK14x 2019-11-25T09:29:07.957 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.970+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.979+0000 7f1386e35ec0 10 manifest: total_size = 5242880 2019-11-25T09:29:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.979+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _QtjwwJWyokczo1QPHbAgGUKxZ7dRYkr 2019-11-25T09:29:07.966 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.979+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-FROZEN-STANDARD state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.979+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:07.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:07.979+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:08.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.091+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:08.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.091+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-25T09:29:08.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.091+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: _QtjwwJWyokczo1QPHbAgGUKxZ7dRYkr 2019-11-25T09:29:08.160 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.173+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:08.173 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.182+0000 7f1386e35ec0 10 manifest: total_size = 5242880 2019-11-25T09:29:08.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.182+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.112 2019-11-25T09:29:08.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.182+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:08.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.182+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:08.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.182+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:08.335 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.348+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:08.335 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.348+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:08.336 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.348+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:08.336 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.348+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-25T09:29:08.336 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.348+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.112 2019-11-25T09:29:08.412 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.424+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:08.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.433+0000 7f1386e35ec0 10 manifest: total_size = 5242880 2019-11-25T09:29:08.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.433+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _hSJ0aEf3SOBpPFeJmfGzmptgAKwzJrr 2019-11-25T09:29:08.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.433+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-FROZEN state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:08.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.433+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:08.421 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.433+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:08.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.544+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:08.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.544+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:08.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.544+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:08.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.544+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-25T09:29:08.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.544+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.15tag is: _hSJ0aEf3SOBpPFeJmfGzmptgAKwzJrr 2019-11-25T09:29:08.651 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.664+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:08.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.673+0000 7f1386e35ec0 10 manifest: total_size = 5242880 2019-11-25T09:29:08.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.673+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.114 2019-11-25T09:29:08.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.673+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-LUKEWARM state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:08.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.673+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:08.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.673+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:08.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.826+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:08.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.826+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:08.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.826+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:08.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.826+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-25T09:29:08.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.826+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.114 2019-11-25T09:29:08.898 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.911+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:08.907 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.920+0000 7f1386e35ec0 10 manifest: total_size = 5242880 2019-11-25T09:29:08.907 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.920+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _j2oy8VDr3xbb__AzO3-dyuQ9R1QbZQJ 2019-11-25T09:29:08.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.920+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-LUKEWARM-STANDARD state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:08.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.920+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:08.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:08.920+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:09.015 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.028+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:09.015 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.028+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=0 rule->part_size=0 2019-11-25T09:29:09.015 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.028+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _j2oy8VDr3xbb__AzO3-dyuQ9R1QbZQJ 2019-11-25T09:29:09.111 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.122+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:09.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.140+0000 7f1386e35ec0 10 manifest: total_size = 22651 2019-11-25T09:29:09.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.140+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.116 2019-11-25T09:29:09.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.140+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:09.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.140+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:09.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.140+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:09.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.393+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:09.409 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.422+0000 7f1386e35ec0 10 manifest: total_size = 22651 2019-11-25T09:29:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.422+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _14ycp4Su54dYzT5L1sJurUdxz-Q2YXy 2019-11-25T09:29:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.422+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-FROZEN state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.422+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:09.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.422+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:09.633 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.646+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:09.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.646+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=22651 stripe_ofs=22651 part_ofs=0 rule->part_size=0 2019-11-25T09:29:09.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.646+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.19tag is: _14ycp4Su54dYzT5L1sJurUdxz-Q2YXy 2019-11-25T09:29:09.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.741+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x556bbc856298 s->prefetch_data=0 2019-11-25T09:29:09.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.752+0000 7f1386e35ec0 10 manifest: total_size = 22651 2019-11-25T09:29:09.740 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.752+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _YGx0-T4jxziGGdW1NympQs240w1eB1L 2019-11-25T09:29:09.740 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.752+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-LUKEWARM state=0x556bbc856298 s->prefetch_data=0 2019-11-25T09:29:09.740 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.752+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc856298 2019-11-25T09:29:09.741 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.752+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:09.894 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.906+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:09.894 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.906+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=22651 stripe_ofs=22651 part_ofs=0 rule->part_size=0 2019-11-25T09:29:09.894 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.906+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.9tag is: _YGx0-T4jxziGGdW1NympQs240w1eB1L 2019-11-25T09:29:09.971 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.984+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:09.983 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.996+0000 7f1386e35ec0 10 manifest: total_size = 22651 2019-11-25T09:29:09.983 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.996+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _AlQy3UUQGYOc6KrXDnQqJlHdHt5t6hq 2019-11-25T09:29:09.983 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.996+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-copy.1:foo-STANDARD-STANDARD state=0x556bbc851fb8 s->prefetch_data=0 2019-11-25T09:29:09.983 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.996+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc851fb8 2019-11-25T09:29:09.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:09.996+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 2019-11-25T09:29:10.144 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.149+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-copy.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7]) start _multipart_[] num_entries 1001 2019-11-25T09:29:10.145 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.149+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:10.244 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.257+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-copy.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:10.325 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.338+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-copy.1 state=0x556bbc848c20 s->prefetch_data=0 2019-11-25T09:29:10.325 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.338+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:10.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.354+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 info.flags=0x16 2019-11-25T09:29:10.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.354+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 to cache LRU end 2019-11-25T09:29:10.343 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.354+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:10.343 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.354+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:10.343 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.354+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:10.343 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.354+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-copy.1 from cache 2019-11-25T09:29:10.344 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.355+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=189 2019-11-25T09:29:10.351 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.364+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584972 cookie 93921212046944 notifier 4499 bl.length()=189 2019-11-25T09:29:10.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.646+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7 from cache 2019-11-25T09:29:10.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.646+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=246 2019-11-25T09:29:10.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:10.666+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 93921212049840 notifier 4499 bl.length()=246 2019-11-25T09:29:11.306 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.319+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x556bbc848c20 s->prefetch_data=0 2019-11-25T09:29:11.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.319+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : miss 2019-11-25T09:29:11.323 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.335+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x6 2019-11-25T09:29:11.323 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.335+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-11-25T09:29:11.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.335+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:11.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.335+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:11.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.335+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:11.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.335+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.349+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=237 2019-11-25T09:29:11.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.349+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x1 2019-11-25T09:29:11.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.349+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-11-25T09:29:11.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.349+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 state=0x556bbc795bf0 s->prefetch_data=0 2019-11-25T09:29:11.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.349+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 : miss 2019-11-25T09:29:11.355 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 info.flags=0x16 2019-11-25T09:29:11.355 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 to cache LRU end 2019-11-25T09:29:11.356 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:11.356 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:11.356 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:11.356 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:11.357 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:11.357 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:11.357 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.368+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:11.379 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.392+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=329 2019-11-25T09:29:11.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.392+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 info.flags=0x13 2019-11-25T09:29:11.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.392+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 to cache LRU end 2019-11-25T09:29:11.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.392+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:11.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.392+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:11.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.392+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:11.407 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.420+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9]) start [] num_entries 1100 2019-11-25T09:29:11.487 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-11-25T09:29:11.487 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-FROZEN[] 2019-11-25T09:29:11.487 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-LUKEWARM[] 2019-11-25T09:29:11.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN-STANDARD[] 2019-11-25T09:29:11.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-11-25T09:29:11.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-FROZEN[] 2019-11-25T09:29:11.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-LUKEWARM[] 2019-11-25T09:29:11.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM-STANDARD[] 2019-11-25T09:29:11.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-11-25T09:29:11.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-FROZEN[] 2019-11-25T09:29:11.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-LUKEWARM[] 2019-11-25T09:29:11.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD-STANDARD[] 2019-11-25T09:29:11.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.499+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:11.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.520+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:11.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.520+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.224 2019-11-25T09:29:11.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.520+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:11.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.520+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8533f8 2019-11-25T09:29:11.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.520+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:11.680 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:11.681 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:11.681 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:11.681 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:11.681 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:11.681 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:11.681 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:11.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:11.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:11.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:11.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-25T09:29:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-25T09:29:11.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.692+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.14tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.224 2019-11-25T09:29:11.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.746+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:11.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.762+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:11.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.762+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.225 2019-11-25T09:29:11.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.762+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-FROZEN state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:11.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.762+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8533f8 2019-11-25T09:29:11.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.763+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:11.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:11.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:11.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:11.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:11.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:11.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:11.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:11.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:11.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:11.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:11.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:11.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:11.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-25T09:29:11.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:11.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-25T09:29:11.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.903+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.225 2019-11-25T09:29:11.953 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.956+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:11.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.967+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:11.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.967+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _MbZUKTZE1RYYKYtajIXiNUvClsmkPTi 2019-11-25T09:29:11.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.967+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-LUKEWARM state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:11.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.967+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8533f8 2019-11-25T09:29:11.956 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:11.967+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:12.075 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.087+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:12.075 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.087+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:12.076 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.087+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:12.076 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.087+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-25T09:29:12.076 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.087+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:12.076 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.087+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-11-25T09:29:12.076 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.087+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.30tag is: _MbZUKTZE1RYYKYtajIXiNUvClsmkPTi 2019-11-25T09:29:12.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.129+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:12.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.140+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:12.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.140+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _J714CAhrvbaK4jNY3N7AIx2hlfhYvyn 2019-11-25T09:29:12.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.140+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-FROZEN-STANDARD state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:12.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.140+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8533f8 2019-11-25T09:29:12.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.140+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:12.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.287+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:12.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.287+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-25T09:29:12.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.287+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:12.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.287+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-11-25T09:29:12.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.287+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: _J714CAhrvbaK4jNY3N7AIx2hlfhYvyn 2019-11-25T09:29:12.315 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.327+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:12.325 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.338+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:12.326 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.338+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.233 2019-11-25T09:29:12.326 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.338+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:12.326 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.338+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8533f8 2019-11-25T09:29:12.327 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.338+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:12.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:12.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:12.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:12.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:12.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:12.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:12.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:12.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:12.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:12.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:12.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:12.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:12.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-25T09:29:12.483 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:12.483 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-25T09:29:12.483 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.488+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.233 2019-11-25T09:29:12.522 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.535+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:12.539 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.552+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:12.539 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.552+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _DmCHlKT-MIVQ2KWqL-HE0UI4MRIlbiI 2019-11-25T09:29:12.540 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.552+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-FROZEN state=0x556bbc8533f8 s->prefetch_data=0 2019-11-25T09:29:12.540 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.552+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8533f8 2019-11-25T09:29:12.540 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.552+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:12.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.716+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:12.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.716+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:12.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.716+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:12.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.716+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-25T09:29:12.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.716+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:12.704 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.716+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-11-25T09:29:12.704 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.716+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: _DmCHlKT-MIVQ2KWqL-HE0UI4MRIlbiI 2019-11-25T09:29:12.745 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.758+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:12.761 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.769+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:12.761 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.769+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.235 2019-11-25T09:29:12.762 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.769+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-LUKEWARM state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:12.762 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.769+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852168 2019-11-25T09:29:12.762 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.769+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:12.942 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:12.943 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:12.943 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:12.943 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:12.943 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:12.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:12.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:12.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:12.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:12.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:12.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:12.945 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:12.945 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-25T09:29:12.945 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:12.945 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-25T09:29:12.945 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:12.955+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.31tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.235 2019-11-25T09:29:13.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.018+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.017 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.030+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:13.017 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.030+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _Pug5muhZYUp9Sg8KtEVANLcFSNBGpgf 2019-11-25T09:29:13.017 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.030+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-LUKEWARM-STANDARD state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.017 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.030+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852168 2019-11-25T09:29:13.017 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.030+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:13.210 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.223+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:13.211 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.223+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-25T09:29:13.211 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.223+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:13.211 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.223+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=0 rule->part_size=0 2019-11-25T09:29:13.211 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.223+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.6tag is: _Pug5muhZYUp9Sg8KtEVANLcFSNBGpgf 2019-11-25T09:29:13.267 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.280+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.280 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.292+0000 7f1386e35ec0 10 manifest: total_size = 1140 2019-11-25T09:29:13.280 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.292+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.242 2019-11-25T09:29:13.280 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.292+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.280 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.292+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852168 2019-11-25T09:29:13.280 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.293+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:13.436 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-25T09:29:13.436 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-11-25T09:29:13.436 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-11-25T09:29:13.436 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-25T09:29:13.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-11-25T09:29:13.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=110 2019-11-25T09:29:13.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=110 rules.size()=2 2019-11-25T09:29:13.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=110 2019-11-25T09:29:13.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=1140 rule->part_size=110 2019-11-25T09:29:13.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.448+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.21tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.242 2019-11-25T09:29:13.499 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.512+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.509 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.522+0000 7f1386e35ec0 10 manifest: total_size = 1140 2019-11-25T09:29:13.509 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.522+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _R_VLqKNLl8LO1nzd_y1Jb3LhdSWbobE 2019-11-25T09:29:13.510 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.522+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-FROZEN state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.510 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.522+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852168 2019-11-25T09:29:13.510 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.522+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:13.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.637+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:13.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.637+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=0 rule->part_size=0 2019-11-25T09:29:13.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.637+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.22tag is: _R_VLqKNLl8LO1nzd_y1Jb3LhdSWbobE 2019-11-25T09:29:13.707 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.720+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.724 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.733+0000 7f1386e35ec0 10 manifest: total_size = 1140 2019-11-25T09:29:13.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.733+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to _rNr6UH_m01Fwy_PokqYmu8J0fE91Tf9 2019-11-25T09:29:13.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.733+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-LUKEWARM state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.733+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852168 2019-11-25T09:29:13.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.733+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:13.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.899+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:13.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.899+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=0 rule->part_size=0 2019-11-25T09:29:13.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.899+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.17tag is: _rNr6UH_m01Fwy_PokqYmu8J0fE91Tf9 2019-11-25T09:29:13.942 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.951+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.949 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.962+0000 7f1386e35ec0 10 manifest: total_size = 1140 2019-11-25T09:29:13.949 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.962+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.245 2019-11-25T09:29:13.950 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.962+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart-copy.1:foo-STANDARD-STANDARD state=0x556bbc852168 s->prefetch_data=0 2019-11-25T09:29:13.950 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.962+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852168 2019-11-25T09:29:13.950 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:13.962+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 2019-11-25T09:29:14.102 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-25T09:29:14.102 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=515 2019-11-25T09:29:14.102 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=515 stripe_ofs=515 part_ofs=515 rule->part_size=515 2019-11-25T09:29:14.102 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=515 rules.size()=2 2019-11-25T09:29:14.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194819 part_ofs=515 rule->part_size=515 2019-11-25T09:29:14.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1030 stripe_ofs=1030 part_ofs=1030 rule->part_size=110 2019-11-25T09:29:14.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=110 rules.size()=2 2019-11-25T09:29:14.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4195334 part_ofs=1030 rule->part_size=110 2019-11-25T09:29:14.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=1140 stripe_ofs=1140 part_ofs=1140 rule->part_size=110 2019-11-25T09:29:14.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.115+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.245 2019-11-25T09:29:14.179 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.192+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart-copy.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9]) start _multipart_[] num_entries 1001 2019-11-25T09:29:14.180 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.192+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:14.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.288+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart-copy.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:14.398 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.411+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart-copy.1 state=0x556bbc796500 s->prefetch_data=0 2019-11-25T09:29:14.399 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.411+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:14.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.432+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 info.flags=0x16 2019-11-25T09:29:14.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.432+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 to cache LRU end 2019-11-25T09:29:14.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.432+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:14.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.432+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:14.421 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.432+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:14.421 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.432+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart-copy.1 from cache 2019-11-25T09:29:14.421 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.432+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=199 2019-11-25T09:29:14.436 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.449+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584963 cookie 93921212048640 notifier 4499 bl.length()=199 2019-11-25T09:29:14.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.694+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart-copy.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9 from cache 2019-11-25T09:29:14.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.695+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=256 2019-11-25T09:29:14.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:14.740+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584960 cookie 93921212056320 notifier 4499 bl.length()=256 2019-11-25T09:29:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.258+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x556bbc856550 s->prefetch_data=0 2019-11-25T09:29:15.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.258+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : miss 2019-11-25T09:29:15.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.286+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x6 2019-11-25T09:29:15.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.286+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-11-25T09:29:15.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.286+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:15.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.286+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:15.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.286+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:15.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.286+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:15.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.303+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=232 2019-11-25T09:29:15.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.303+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x1 2019-11-25T09:29:15.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.303+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-11-25T09:29:15.292 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.303+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 state=0x556bbc795bf0 s->prefetch_data=0 2019-11-25T09:29:15.292 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.303+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 : miss 2019-11-25T09:29:15.323 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 info.flags=0x16 2019-11-25T09:29:15.323 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 to cache LRU end 2019-11-25T09:29:15.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:15.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:15.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:15.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:15.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:15.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:15.325 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.334+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:15.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.362+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=324 2019-11-25T09:29:15.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.362+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 info.flags=0x13 2019-11-25T09:29:15.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.362+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 to cache LRU end 2019-11-25T09:29:15.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.362+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:15.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.362+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:15.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.362+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 2019-11-25T09:29:15.400 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.412+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class-multipart.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8]) start [] num_entries 1100 2019-11-25T09:29:15.456 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.469+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-11-25T09:29:15.457 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.469+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-11-25T09:29:15.457 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.469+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-11-25T09:29:15.457 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.469+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x556bbc852698 s->prefetch_data=0 2019-11-25T09:29:15.474 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.486+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:15.474 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.486+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.196 2019-11-25T09:29:15.474 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.486+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-FROZEN state=0x556bbc852698 s->prefetch_data=0 2019-11-25T09:29:15.474 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.486+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852698 2019-11-25T09:29:15.474 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.486+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 2019-11-25T09:29:15.613 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:15.613 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:15.613 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:15.614 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:15.614 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:15.614 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:15.614 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:15.614 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:15.614 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:15.615 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:15.615 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:15.615 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:15.615 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-25T09:29:15.615 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:15.615 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-25T09:29:15.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.626+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.0tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.196 2019-11-25T09:29:15.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.708+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x556bbc852698 s->prefetch_data=0 2019-11-25T09:29:15.708 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.721+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:15.708 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.721+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.203 2019-11-25T09:29:15.708 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.721+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-LUKEWARM state=0x556bbc852698 s->prefetch_data=0 2019-11-25T09:29:15.709 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.721+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852698 2019-11-25T09:29:15.709 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.721+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 2019-11-25T09:29:15.883 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:15.884 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:15.884 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:15.884 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:15.884 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:15.884 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:15.885 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:15.885 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:15.885 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:15.885 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:15.885 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:15.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:15.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-25T09:29:15.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:15.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-25T09:29:15.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.896+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.23tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.203 2019-11-25T09:29:15.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.968+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x556bbc852698 s->prefetch_data=0 2019-11-25T09:29:15.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.982+0000 7f1386e35ec0 10 manifest: total_size = 11534336 2019-11-25T09:29:15.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.982+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.210 2019-11-25T09:29:15.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.982+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class-multipart.1:foo-STANDARD state=0x556bbc852698 s->prefetch_data=0 2019-11-25T09:29:15.970 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.982+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc852698 2019-11-25T09:29:15.970 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:15.982+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 2019-11-25T09:29:16.115 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:16.115 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:16.116 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:16.116 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:16.116 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=8388608 part_ofs=0 rule->part_size=5242880 2019-11-25T09:29:16.116 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=5242880 stripe_ofs=5242880 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:16.116 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:16.116 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:16.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=9437184 stripe_ofs=9437184 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:16.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=5242880 rules.size()=2 2019-11-25T09:29:16.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=13631488 part_ofs=5242880 rule->part_size=5242880 2019-11-25T09:29:16.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:16.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=1048576 rules.size()=2 2019-11-25T09:29:16.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): stripe_ofs=14680064 part_ofs=10485760 rule->part_size=1048576 2019-11-25T09:29:16.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=11534336 stripe_ofs=11534336 part_ofs=11534336 rule->part_size=1048576 2019-11-25T09:29:16.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.128+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.210 2019-11-25T09:29:16.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.182+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class-multipart.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8]) start _multipart_[] num_entries 1001 2019-11-25T09:29:16.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.182+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:16.208 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.221+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class-multipart.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:16.301 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.313+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class-multipart.1 state=0x556bbc856550 s->prefetch_data=0 2019-11-25T09:29:16.301 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.313+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:16.317 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.330+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 info.flags=0x16 2019-11-25T09:29:16.317 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.330+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 to cache LRU end 2019-11-25T09:29:16.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.330+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:16.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.330+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:16.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.330+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:16.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.330+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class-multipart.1 from cache 2019-11-25T09:29:16.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.330+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=194 2019-11-25T09:29:16.331 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.344+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 93921212049840 notifier 4499 bl.length()=194 2019-11-25T09:29:16.694 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.707+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class-multipart.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8 from cache 2019-11-25T09:29:16.694 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.707+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=251 2019-11-25T09:29:16.709 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:16.722+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584961 cookie 93921212056320 notifier 4499 bl.length()=251 2019-11-25T09:29:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.290+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x556bbc795bf0 s->prefetch_data=0 2019-11-25T09:29:17.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.290+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : miss 2019-11-25T09:29:17.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.304+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x6 2019-11-25T09:29:17.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.304+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-11-25T09:29:17.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.304+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:17.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.304+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:17.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.304+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:17.292 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.304+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:17.301 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.314+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=222 2019-11-25T09:29:17.301 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.314+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x1 2019-11-25T09:29:17.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.314+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-11-25T09:29:17.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.314+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 state=0x556bbc856550 s->prefetch_data=0 2019-11-25T09:29:17.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.314+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 : miss 2019-11-25T09:29:17.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 info.flags=0x16 2019-11-25T09:29:17.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 to cache LRU end 2019-11-25T09:29:17.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:17.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:17.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:17.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:17.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:17.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:17.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.345+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:17.353 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.366+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=314 2019-11-25T09:29:17.353 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.366+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 info.flags=0x13 2019-11-25T09:29:17.353 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.366+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 to cache LRU end 2019-11-25T09:29:17.353 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.366+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:17.354 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.366+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:17.354 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.366+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 2019-11-25T09:29:17.374 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.387+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-obj-storage-class.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6]) start [] num_entries 1100 2019-11-25T09:29:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.424+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-FROZEN[] 2019-11-25T09:29:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.424+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-LUKEWARM[] 2019-11-25T09:29:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.424+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got foo-STANDARD[] 2019-11-25T09:29:17.411 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.424+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:17.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.446+0000 7f1386e35ec0 10 manifest: total_size = 10485760 2019-11-25T09:29:17.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.446+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.84 2019-11-25T09:29:17.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.446+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class.1:foo-FROZEN state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:17.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.446+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:17.434 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.446+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 2019-11-25T09:29:17.602 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.615+0000 7f1344fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:29:17.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.621+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:17.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.621+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:17.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.621+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:17.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.621+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-25T09:29:17.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.621+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:17.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.621+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-11-25T09:29:17.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.621+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.13tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.84 2019-11-25T09:29:17.657 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.670+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:17.670 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.683+0000 7f1386e35ec0 10 manifest: total_size = 10485760 2019-11-25T09:29:17.670 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.683+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.85 2019-11-25T09:29:17.671 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.683+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class.1:foo-LUKEWARM state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:17.671 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.683+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:17.671 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.683+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 2019-11-25T09:29:17.808 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.821+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:17.808 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.821+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=4194304 stripe_ofs=4194304 part_ofs=0 rule->part_size=0 2019-11-25T09:29:17.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.821+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:17.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.821+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=8388608 stripe_ofs=8388608 part_ofs=0 rule->part_size=0 2019-11-25T09:29:17.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.821+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): rule->part_size=0 rules.size()=1 2019-11-25T09:29:17.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.821+0000 7f1386e35ec0 20 RGWObjManifest::operator++(): result: ofs=10485760 stripe_ofs=10485760 part_ofs=0 rule->part_size=0 2019-11-25T09:29:17.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.821+0000 7f1386e35ec0 20 garbage collection: RGWGC::send_chain - on object name: gc.29tag is: 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.85 2019-11-25T09:29:17.901 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.914+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:17.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.927+0000 7f1386e35ec0 10 manifest: total_size = 53502 2019-11-25T09:29:17.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.927+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.86 2019-11-25T09:29:17.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.927+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-obj-storage-class.1:foo-STANDARD state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:17.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.927+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:17.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:17.927+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 2019-11-25T09:29:18.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.057+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-obj-storage-class.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6]) start _multipart_[] num_entries 1001 2019-11-25T09:29:18.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.057+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:18.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.154+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-obj-storage-class.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:18.223 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.236+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-obj-storage-class.1 state=0x556bbc795bf0 s->prefetch_data=0 2019-11-25T09:29:18.223 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.236+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:18.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.252+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 info.flags=0x16 2019-11-25T09:29:18.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.252+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 to cache LRU end 2019-11-25T09:29:18.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.252+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:18.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.252+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:18.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.252+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:18.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.252+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-obj-storage-class.1 from cache 2019-11-25T09:29:18.240 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.252+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=184 2019-11-25T09:29:18.289 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.302+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584973 cookie 93921212046944 notifier 4499 bl.length()=184 2019-11-25T09:29:18.982 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.995+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-obj-storage-class.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6 from cache 2019-11-25T09:29:18.982 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:18.995+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=241 2019-11-25T09:29:19.001 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.015+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584964 cookie 93921212048640 notifier 4499 bl.length()=241 2019-11-25T09:29:19.639 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.652+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x556bbc795bf0 s->prefetch_data=0 2019-11-25T09:29:19.639 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.652+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : miss 2019-11-25T09:29:19.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.672+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x6 2019-11-25T09:29:19.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.672+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-11-25T09:29:19.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.672+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:19.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.672+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:19.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.672+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:19.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.672+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:19.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.692+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=221 2019-11-25T09:29:19.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.692+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x1 2019-11-25T09:29:19.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.692+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-11-25T09:29:19.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.692+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 state=0x556bbc856550 s->prefetch_data=0 2019-11-25T09:29:19.680 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.692+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 : miss 2019-11-25T09:29:19.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 info.flags=0x16 2019-11-25T09:29:19.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 to cache LRU end 2019-11-25T09:29:19.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:19.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:19.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:19.704 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:19.704 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:19.704 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:19.704 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.716+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:19.734 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.747+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=313 2019-11-25T09:29:19.734 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.747+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 info.flags=0x13 2019-11-25T09:29:19.735 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.747+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 to cache LRU end 2019-11-25T09:29:19.735 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.747+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:19.736 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.747+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:19.736 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.747+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 2019-11-25T09:29:19.758 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.771+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10]) start [] num_entries 1100 2019-11-25T09:29:19.795 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.808+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-11-25T09:29:19.795 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.808+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-11-25T09:29:19.795 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.808+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-11-25T09:29:19.795 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.808+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-11-25T09:29:19.796 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.808+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:19.810 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.823+0000 7f1386e35ec0 10 manifest: total_size = 49187 2019-11-25T09:29:19.810 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.823+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.305 2019-11-25T09:29:19.811 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.823+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.1:__ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:19.811 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.823+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:19.811 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.823+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 2019-11-25T09:29:19.973 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:19.980+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:20.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.025+0000 7f1386e35ec0 10 manifest: total_size = 110 2019-11-25T09:29:20.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.025+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.307 2019-11-25T09:29:20.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.025+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.1:__ _ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:20.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.025+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:20.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.025+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 2019-11-25T09:29:20.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.201+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:20.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.219+0000 7f1386e35ec0 10 manifest: total_size = 4134 2019-11-25T09:29:20.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.219+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.306 2019-11-25T09:29:20.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.219+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.1:___ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:20.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.219+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:20.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.219+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 2019-11-25T09:29:20.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.492+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:20.496 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.510+0000 7f1386e35ec0 10 manifest: total_size = 110 2019-11-25T09:29:20.497 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.510+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.304 2019-11-25T09:29:20.497 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.510+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.1:obj state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:20.497 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.510+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:20.497 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.510+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 2019-11-25T09:29:20.740 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.753+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10]) start _multipart_[] num_entries 1001 2019-11-25T09:29:20.740 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.753+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:20.831 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.844+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.1[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:20.938 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.951+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.1 state=0x556bbc795bf0 s->prefetch_data=0 2019-11-25T09:29:20.938 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.951+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:20.957 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.970+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 info.flags=0x16 2019-11-25T09:29:20.957 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.970+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 to cache LRU end 2019-11-25T09:29:20.957 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.970+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:20.958 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.970+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:20.958 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.970+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:20.958 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.970+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.1 from cache 2019-11-25T09:29:20.958 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.970+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-11-25T09:29:20.971 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:20.984+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 93921212045040 notifier 4499 bl.length()=181 2019-11-25T09:29:21.373 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:21.386+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.1:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.10 from cache 2019-11-25T09:29:21.373 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:21.386+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=239 2019-11-25T09:29:21.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:21.399+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584967 cookie 93921212049840 notifier 4499 bl.length()=239 2019-11-25T09:29:22.064 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.077+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x556bbc795bf0 s->prefetch_data=0 2019-11-25T09:29:22.065 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.077+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : miss 2019-11-25T09:29:22.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.094+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x6 2019-11-25T09:29:22.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.094+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-11-25T09:29:22.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.094+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:22.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.094+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:22.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.094+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:22.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.094+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:22.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.107+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=221 2019-11-25T09:29:22.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.107+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x1 2019-11-25T09:29:22.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.107+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-11-25T09:29:22.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.107+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 state=0x556bbc856550 s->prefetch_data=0 2019-11-25T09:29:22.095 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.107+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 : miss 2019-11-25T09:29:22.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 info.flags=0x16 2019-11-25T09:29:22.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 to cache LRU end 2019-11-25T09:29:22.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:22.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:22.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:22.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:22.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:22.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:22.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.126+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:22.134 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.147+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=313 2019-11-25T09:29:22.134 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.147+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 info.flags=0x13 2019-11-25T09:29:22.134 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.147+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 to cache LRU end 2019-11-25T09:29:22.135 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.147+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:22.135 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.147+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:22.135 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.147+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 2019-11-25T09:29:22.187 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.200+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.2[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11]) start [] num_entries 1100 2019-11-25T09:29:22.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.288+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-11-25T09:29:22.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.288+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-11-25T09:29:22.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.288+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-11-25T09:29:22.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.288+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-11-25T09:29:22.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.289+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:22.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.306+0000 7f1386e35ec0 10 manifest: total_size = 24595 2019-11-25T09:29:22.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.306+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.312 2019-11-25T09:29:22.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.306+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.2:__ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:22.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.306+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:22.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.306+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 2019-11-25T09:29:22.512 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.519+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:22.522 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.535+0000 7f1386e35ec0 10 manifest: total_size = 527 2019-11-25T09:29:22.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.535+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.314 2019-11-25T09:29:22.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.535+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.2:__ _ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:22.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.535+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:22.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.535+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 2019-11-25T09:29:22.845 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.858+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:22.856 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.869+0000 7f1386e35ec0 10 manifest: total_size = 24595 2019-11-25T09:29:22.856 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.869+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.313 2019-11-25T09:29:22.856 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.869+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.2:___ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:22.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.869+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:22.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:22.869+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 2019-11-25T09:29:23.145 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.157+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:23.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.171+0000 7f1386e35ec0 10 manifest: total_size = 524288 2019-11-25T09:29:23.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.171+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.311 2019-11-25T09:29:23.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.171+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.2:obj state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:23.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.171+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:23.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.171+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 2019-11-25T09:29:23.330 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.343+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.2[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11]) start _multipart_[] num_entries 1001 2019-11-25T09:29:23.331 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.343+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:23.462 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.474+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.2[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:23.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.565+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.2 state=0x556bbc8567b0 s->prefetch_data=0 2019-11-25T09:29:23.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.565+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:23.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.579+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 info.flags=0x16 2019-11-25T09:29:23.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.579+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 to cache LRU end 2019-11-25T09:29:23.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.579+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:23.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.579+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:23.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.579+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:23.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.579+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.2 from cache 2019-11-25T09:29:23.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.579+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-11-25T09:29:23.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.598+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584968 cookie 93921212045040 notifier 4499 bl.length()=181 2019-11-25T09:29:23.801 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.812+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.2:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.11 from cache 2019-11-25T09:29:23.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.812+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=239 2019-11-25T09:29:23.811 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:23.824+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584969 cookie 93921212049840 notifier 4499 bl.length()=239 2019-11-25T09:29:24.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.249+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x556bbc856550 s->prefetch_data=0 2019-11-25T09:29:24.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.249+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : miss 2019-11-25T09:29:24.256 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.269+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x6 2019-11-25T09:29:24.256 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.269+0000 7f1386e35ec0 10 adding default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-11-25T09:29:24.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.269+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:24.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.269+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:24.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.269+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x1, cached=0x6) 2019-11-25T09:29:24.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.269+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:24.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.286+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=221 2019-11-25T09:29:24.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.286+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x1 2019-11-25T09:29:24.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.286+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-11-25T09:29:24.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.286+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c21580 obj=default.rgw.meta:root:.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 state=0x556bbc84a980 s->prefetch_data=0 2019-11-25T09:29:24.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.286+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 : miss 2019-11-25T09:29:24.298 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 info.flags=0x16 2019-11-25T09:29:24.298 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 10 adding default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 to cache LRU end 2019-11-25T09:29:24.298 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:24.298 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 20 Read xattr: user.rgw.acl 2019-11-25T09:29:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 20 Read xattr: user.rgw.idtag 2019-11-25T09:29:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 : type miss (requested=0x13, cached=0x16) 2019-11-25T09:29:24.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.311+0000 7f1386e35ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:24.348 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.361+0000 7f1386e35ec0 20 rados_obj.operate() r=0 bl.length=313 2019-11-25T09:29:24.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.361+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 info.flags=0x13 2019-11-25T09:29:24.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.361+0000 7f1386e35ec0 10 moving default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 to cache LRU end 2019-11-25T09:29:24.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.361+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:24.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.361+0000 7f1386e35ec0 10 updating xattr: name=user.rgw.acl bl.length()=209 2019-11-25T09:29:24.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.361+0000 7f1386e35ec0 10 chain_cache_entry: cache_locator=default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 2019-11-25T09:29:24.384 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.397+0000 7f1386e35ec0 10 cls_bucket_list_unordered :test-client.0-r-test-small-obj-data.3[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12]) start [] num_entries 1100 2019-11-25T09:29:24.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got __[] 2019-11-25T09:29:24.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got __ _[] 2019-11-25T09:29:24.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got ___[] 2019-11-25T09:29:24.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.442+0000 7f1386e35ec0 10 RGWRados::cls_bucket_list_unordered: got obj[] 2019-11-25T09:29:24.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.442+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.3:__ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:24.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.465+0000 7f1386e35ec0 10 manifest: total_size = 0 2019-11-25T09:29:24.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.465+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.319 2019-11-25T09:29:24.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.465+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.3:__ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:24.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.465+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:24.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.465+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 2019-11-25T09:29:24.595 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.608+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:24.607 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.620+0000 7f1386e35ec0 10 manifest: total_size = 0 2019-11-25T09:29:24.607 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.620+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.321 2019-11-25T09:29:24.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.620+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.3:__ _ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:24.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.620+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:24.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.620+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 2019-11-25T09:29:24.836 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.849+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.3:___ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:24.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.873+0000 7f1386e35ec0 10 manifest: total_size = 0 2019-11-25T09:29:24.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.873+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.320 2019-11-25T09:29:24.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.873+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.3:___ state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:24.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.873+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:24.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:24.873+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 2019-11-25T09:29:25.099 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.112+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:25.125 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.138+0000 7f1386e35ec0 10 manifest: total_size = 0 2019-11-25T09:29:25.125 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.138+0000 7f1386e35ec0 20 get_obj_state: setting s->obj_tag to 103c5a94-eb69-4977-8c90-9f6d634d8761.4500.318 2019-11-25T09:29:25.125 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.138+0000 7f1386e35ec0 20 get_obj_state: rctx=0x7ffe53c215a0 obj=test-client.0-r-test-small-obj-data.3:obj state=0x556bbc8522c8 s->prefetch_data=0 2019-11-25T09:29:25.125 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.138+0000 7f1386e35ec0 20 prepare_atomic_modification: state is not atomic. state=0x556bbc8522c8 2019-11-25T09:29:25.125 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.138+0000 7f1386e35ec0 20 bucket index object: default.rgw.buckets.index:.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 2019-11-25T09:29:25.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.423+0000 7f1386e35ec0 10 cls_bucket_list_ordered :test-client.0-r-test-small-obj-data.3[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12]) start _multipart_[] num_entries 1001 2019-11-25T09:29:25.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.423+0000 7f1386e35ec0 10 cls_bucket_list_ordered request from each of 1 shard(s) for 1002 entries to get 1001 total entries 2019-11-25T09:29:25.509 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.523+0000 7f1386e35ec0 20 abort_bucket_multiparts INFO: aborting and cleaning up multipart upload(s); bucket=":test-client.0-r-test-small-obj-data.3[103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12])"; objs.size()=0; is_truncated=0 2019-11-25T09:29:25.610 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.623+0000 7f1386e35ec0 20 get_system_obj_state: rctx=0x7ffe53c20f38 obj=default.rgw.meta:root:test-client.0-r-test-small-obj-data.3 state=0x556bbc852bb0 s->prefetch_data=0 2019-11-25T09:29:25.610 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.623+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : type miss (requested=0x16, cached=0x3) 2019-11-25T09:29:25.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.642+0000 7f1386e35ec0 10 cache put: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 info.flags=0x16 2019-11-25T09:29:25.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.642+0000 7f1386e35ec0 10 moving default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 to cache LRU end 2019-11-25T09:29:25.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.642+0000 7f1386e35ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T09:29:25.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.642+0000 7f1386e35ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T09:29:25.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.642+0000 7f1386e35ec0 10 cache get: name=default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 : hit (requested=0x11, cached=0x17) 2019-11-25T09:29:25.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.642+0000 7f1386e35ec0 10 removing default.rgw.meta+root+test-client.0-r-test-small-obj-data.3 from cache 2019-11-25T09:29:25.630 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.642+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=181 2019-11-25T09:29:25.638 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.652+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584970 cookie 93921212045040 notifier 4499 bl.length()=181 2019-11-25T09:29:25.700 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.713+0000 7f1386e35ec0 10 removing default.rgw.meta+root+.bucket.meta.test-client.0-r-test-small-obj-data.3:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.12 from cache 2019-11-25T09:29:25.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.713+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=239 2019-11-25T09:29:25.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:25.723+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584971 cookie 93921212049840 notifier 4499 bl.length()=239 2019-11-25T09:29:26.077 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.090+0000 7f1386e35ec0 10 removing key index: CZHRSUPRPHZWHQIHEBMH 2019-11-25T09:29:26.077 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.090+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=170 2019-11-25T09:29:26.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.100+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584965 cookie 93921212048640 notifier 4499 bl.length()=170 2019-11-25T09:29:26.344 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.357+0000 7f1386e35ec0 10 removing email index: ragweed.client.0+test@test.test 2019-11-25T09:29:26.344 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.357+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=182 2019-11-25T09:29:26.357 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.370+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584974 cookie 93921212046944 notifier 4499 bl.length()=182 2019-11-25T09:29:26.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.936+0000 7f1386e35ec0 10 removing user buckets index 2019-11-25T09:29:26.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.936+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=173 2019-11-25T09:29:26.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:26.954+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584972 cookie 93921212049840 notifier 4499 bl.length()=173 2019-11-25T09:29:27.292 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.305+0000 7f1386e35ec0 10 removing user index: ragweed.client.0 2019-11-25T09:29:27.292 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.305+0000 7f1386e35ec0 10 removing default.rgw.meta+users.uid+ragweed.client.0 from cache 2019-11-25T09:29:27.292 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.305+0000 7f1386e35ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=165 2019-11-25T09:29:27.313 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.327+0000 7f1369ffb700 10 RGWWatcher::handle_notify() notify_id 579820584975 cookie 93921212052048 notifier 4499 bl.length()=165 2019-11-25T09:29:27.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.494+0000 7f1386e35ec0 20 remove_watcher() i=0 2019-11-25T09:29:27.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.494+0000 7f1386e35ec0 2 removed watcher, disabling cache 2019-11-25T09:29:27.540 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.553+0000 7f1386e35ec0 20 remove_watcher() i=1 2019-11-25T09:29:27.593 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.607+0000 7f1386e35ec0 20 remove_watcher() i=2 2019-11-25T09:29:27.630 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.644+0000 7f1386e35ec0 20 remove_watcher() i=3 2019-11-25T09:29:27.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.692+0000 7f1386e35ec0 20 remove_watcher() i=4 2019-11-25T09:29:27.718 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.732+0000 7f1386e35ec0 20 remove_watcher() i=5 2019-11-25T09:29:27.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.773+0000 7f1386e35ec0 20 remove_watcher() i=6 2019-11-25T09:29:27.796 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:27.809+0000 7f1386e35ec0 20 remove_watcher() i=7 2019-11-25T09:29:27.803 INFO:tasks.ragweed:Removing ragweed... 2019-11-25T09:29:27.803 INFO:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/ragweed 2019-11-25T09:29:27.887 INFO:teuthology.run_tasks:Running task s3tests-java... 2019-11-25T09:29:27.902 DEBUG:tasks.s3tests_java:S3 Tests Java: __INIT__ 2019-11-25T09:29:27.903 DEBUG:tasks.s3tests_java:S3 Tests Java: SETUP 2019-11-25T09:29:27.903 INFO:tasks.s3tests_java:S3 Tests Java: Downloading test suite... 2019-11-25T09:29:27.903 INFO:teuthology.orchestra.run.smithi153:> git clone -b master https://github.com/ceph/java_s3tests.git /home/ubuntu/cephtest/s3-tests-java 2019-11-25T09:29:27.948 INFO:teuthology.orchestra.run.smithi153.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests-java'... 2019-11-25T09:29:31.796 INFO:tasks.s3tests_java:S3 Tests Java: Installing required packages... 2019-11-25T09:29:31.796 INFO:teuthology.orchestra.run.smithi153:> /home/ubuntu/cephtest/s3-tests-java/bootstrap.sh 2019-11-25T09:29:34.383 INFO:teuthology.orchestra.run.smithi153.stdout: Gradle will be installed under /opt/gradle 2019-11-25T09:29:34.437 INFO:teuthology.orchestra.run.smithi153.stderr:--2019-11-25 09:29:34-- https://services.gradle.org/distributions/gradle-4.7-bin.zip 2019-11-25T09:29:34.443 INFO:teuthology.orchestra.run.smithi153.stderr:Resolving services.gradle.org (services.gradle.org)... 104.18.191.9, 104.18.190.9, 2606:4700::6812:bf09, ... 2019-11-25T09:29:34.451 INFO:teuthology.orchestra.run.smithi153.stderr:Connecting to services.gradle.org (services.gradle.org)|104.18.191.9|:443... connected. 2019-11-25T09:29:34.488 INFO:teuthology.orchestra.run.smithi153.stderr:HTTP request sent, awaiting response... 301 Moved Permanently 2019-11-25T09:29:34.488 INFO:teuthology.orchestra.run.smithi153.stderr:Location: https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip [following] 2019-11-25T09:29:34.488 INFO:teuthology.orchestra.run.smithi153.stderr:--2019-11-25 09:29:34-- https://downloads.gradle-dn.com/distributions/gradle-4.7-bin.zip 2019-11-25T09:29:34.489 INFO:teuthology.orchestra.run.smithi153.stderr:Resolving downloads.gradle-dn.com (downloads.gradle-dn.com)... 104.17.160.20, 104.17.159.20, 2606:4700::6811:a014, ... 2019-11-25T09:29:34.496 INFO:teuthology.orchestra.run.smithi153.stderr:Connecting to downloads.gradle-dn.com (downloads.gradle-dn.com)|104.17.160.20|:443... connected. 2019-11-25T09:29:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:HTTP request sent, awaiting response... 200 OK 2019-11-25T09:29:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:Length: 75323957 (72M) [application/zip] 2019-11-25T09:29:34.532 INFO:teuthology.orchestra.run.smithi153.stderr:Saving to: ‘gradle-4.7-bin.zip’ 2019-11-25T09:29:34.532 INFO:teuthology.orchestra.run.smithi153.stderr: 2019-11-25T09:29:34.541 INFO:teuthology.orchestra.run.smithi153.stderr: 0K .......... .......... .......... .......... .......... 0% 6.71M 11s 2019-11-25T09:29:34.542 INFO:teuthology.orchestra.run.smithi153.stderr: 50K .......... .......... .......... .......... .......... 0% 122M 6s 2019-11-25T09:29:34.549 INFO:teuthology.orchestra.run.smithi153.stderr: 100K .......... .......... .......... .......... .......... 0% 7.04M 7s 2019-11-25T09:29:34.549 INFO:teuthology.orchestra.run.smithi153.stderr: 150K .......... .......... .......... .......... .......... 0% 83.2M 6s 2019-11-25T09:29:34.549 INFO:teuthology.orchestra.run.smithi153.stderr: 200K .......... .......... .......... .......... .......... 0% 118M 5s 2019-11-25T09:29:34.556 INFO:teuthology.orchestra.run.smithi153.stderr: 250K .......... .......... .......... .......... .......... 0% 7.38M 5s 2019-11-25T09:29:34.556 INFO:teuthology.orchestra.run.smithi153.stderr: 300K .......... .......... .......... .......... .......... 0% 106M 5s 2019-11-25T09:29:34.557 INFO:teuthology.orchestra.run.smithi153.stderr: 350K .......... .......... .......... .......... .......... 0% 93.7M 4s 2019-11-25T09:29:34.562 INFO:teuthology.orchestra.run.smithi153.stderr: 400K .......... .......... .......... .......... .......... 0% 90.6M 4s 2019-11-25T09:29:34.562 INFO:teuthology.orchestra.run.smithi153.stderr: 450K .......... .......... .......... .......... .......... 0% 104M 4s 2019-11-25T09:29:34.563 INFO:teuthology.orchestra.run.smithi153.stderr: 500K .......... .......... .......... .......... .......... 0% 117M 3s 2019-11-25T09:29:34.563 INFO:teuthology.orchestra.run.smithi153.stderr: 550K .......... .......... .......... .......... .......... 0% 95.2M 3s 2019-11-25T09:29:34.563 INFO:teuthology.orchestra.run.smithi153.stderr: 600K .......... .......... .......... .......... .......... 0% 109M 3s 2019-11-25T09:29:34.563 INFO:teuthology.orchestra.run.smithi153.stderr: 650K .......... .......... .......... .......... .......... 0% 121M 3s 2019-11-25T09:29:35.785 INFO:teuthology.orchestra.run.smithi153.stderr: 700K .......... .......... .......... .......... .......... 1% 114M 3s 2019-11-25T09:29:35.786 INFO:teuthology.orchestra.run.smithi153.stderr: 750K .......... .......... .......... .......... .......... 1% 11.7M 3s 2019-11-25T09:29:35.786 INFO:teuthology.orchestra.run.smithi153.stderr: 800K .......... .......... .......... .......... .......... 1% 77.3M 3s 2019-11-25T09:29:35.786 INFO:teuthology.orchestra.run.smithi153.stderr: 850K .......... .......... .......... .......... .......... 1% 92.8M 3s 2019-11-25T09:29:35.786 INFO:teuthology.orchestra.run.smithi153.stderr: 900K .......... .......... .......... .......... .......... 1% 110M 2s 2019-11-25T09:29:35.786 INFO:teuthology.orchestra.run.smithi153.stderr: 950K .......... .......... .......... .......... .......... 1% 102M 2s 2019-11-25T09:29:35.786 INFO:teuthology.orchestra.run.smithi153.stderr: 1000K .......... .......... .......... .......... .......... 1% 108M 2s 2019-11-25T09:29:35.786 INFO:teuthology.orchestra.run.smithi153.stderr: 1050K .......... .......... .......... .......... .......... 1% 112M 2s 2019-11-25T09:29:35.787 INFO:teuthology.orchestra.run.smithi153.stderr: 1100K .......... .......... .......... .......... .......... 1% 146M 2s 2019-11-25T09:29:35.787 INFO:teuthology.orchestra.run.smithi153.stderr: 1150K .......... .......... .......... .......... .......... 1% 158M 2s 2019-11-25T09:29:35.787 INFO:teuthology.orchestra.run.smithi153.stderr: 1200K .......... .......... .......... .......... .......... 1% 182M 2s 2019-11-25T09:29:35.787 INFO:teuthology.orchestra.run.smithi153.stderr: 1250K .......... .......... .......... .......... .......... 1% 16.2M 2s 2019-11-25T09:29:35.787 INFO:teuthology.orchestra.run.smithi153.stderr: 1300K .......... .......... .......... .......... .......... 1% 98.0M 2s 2019-11-25T09:29:35.787 INFO:teuthology.orchestra.run.smithi153.stderr: 1350K .......... .......... .......... .......... .......... 1% 103M 2s 2019-11-25T09:29:35.787 INFO:teuthology.orchestra.run.smithi153.stderr: 1400K .......... .......... .......... .......... .......... 1% 126M 2s 2019-11-25T09:29:35.788 INFO:teuthology.orchestra.run.smithi153.stderr: 1450K .......... .......... .......... .......... .......... 2% 121M 2s 2019-11-25T09:29:35.788 INFO:teuthology.orchestra.run.smithi153.stderr: 1500K .......... .......... .......... .......... .......... 2% 136M 2s 2019-11-25T09:29:35.788 INFO:teuthology.orchestra.run.smithi153.stderr: 1550K .......... .......... .......... .......... .......... 2% 103M 2s 2019-11-25T09:29:35.788 INFO:teuthology.orchestra.run.smithi153.stderr: 1600K .......... .......... .......... .......... .......... 2% 97.9M 2s 2019-11-25T09:29:35.788 INFO:teuthology.orchestra.run.smithi153.stderr: 1650K .......... .......... .......... .......... .......... 2% 79.0M 2s 2019-11-25T09:29:35.788 INFO:teuthology.orchestra.run.smithi153.stderr: 1700K .......... .......... .......... .......... .......... 2% 122M 2s 2019-11-25T09:29:35.788 INFO:teuthology.orchestra.run.smithi153.stderr: 1750K .......... .......... .......... .......... .......... 2% 138M 2s 2019-11-25T09:29:35.789 INFO:teuthology.orchestra.run.smithi153.stderr: 1800K .......... .......... .......... .......... .......... 2% 179M 2s 2019-11-25T09:29:35.789 INFO:teuthology.orchestra.run.smithi153.stderr: 1850K .......... .......... .......... .......... .......... 2% 155M 2s 2019-11-25T09:29:35.789 INFO:teuthology.orchestra.run.smithi153.stderr: 1900K .......... .......... .......... .......... .......... 2% 153M 2s 2019-11-25T09:29:35.789 INFO:teuthology.orchestra.run.smithi153.stderr: 1950K .......... .......... .......... .......... .......... 2% 124M 2s 2019-11-25T09:29:35.789 INFO:teuthology.orchestra.run.smithi153.stderr: 2000K .......... .......... .......... .......... .......... 2% 172M 2s 2019-11-25T09:29:35.789 INFO:teuthology.orchestra.run.smithi153.stderr: 2050K .......... .......... .......... .......... .......... 2% 172M 1s 2019-11-25T09:29:35.789 INFO:teuthology.orchestra.run.smithi153.stderr: 2100K .......... .......... .......... .......... .......... 2% 200M 1s 2019-11-25T09:29:35.790 INFO:teuthology.orchestra.run.smithi153.stderr: 2150K .......... .......... .......... .......... .......... 2% 174M 1s 2019-11-25T09:29:35.790 INFO:teuthology.orchestra.run.smithi153.stderr: 2200K .......... .......... .......... .......... .......... 3% 179M 1s 2019-11-25T09:29:35.790 INFO:teuthology.orchestra.run.smithi153.stderr: 2250K .......... .......... .......... .......... .......... 3% 150M 1s 2019-11-25T09:29:35.790 INFO:teuthology.orchestra.run.smithi153.stderr: 2300K .......... .......... .......... .......... .......... 3% 153M 1s 2019-11-25T09:29:35.790 INFO:teuthology.orchestra.run.smithi153.stderr: 2350K .......... .......... .......... .......... .......... 3% 138M 1s 2019-11-25T09:29:35.790 INFO:teuthology.orchestra.run.smithi153.stderr: 2400K .......... .......... .......... .......... .......... 3% 175M 1s 2019-11-25T09:29:35.790 INFO:teuthology.orchestra.run.smithi153.stderr: 2450K .......... .......... .......... .......... .......... 3% 173M 1s 2019-11-25T09:29:35.791 INFO:teuthology.orchestra.run.smithi153.stderr: 2500K .......... .......... .......... .......... .......... 3% 195M 1s 2019-11-25T09:29:35.791 INFO:teuthology.orchestra.run.smithi153.stderr: 2550K .......... .......... .......... .......... .......... 3% 164M 1s 2019-11-25T09:29:35.791 INFO:teuthology.orchestra.run.smithi153.stderr: 2600K .......... .......... .......... .......... .......... 3% 188M 1s 2019-11-25T09:29:35.791 INFO:teuthology.orchestra.run.smithi153.stderr: 2650K .......... .......... .......... .......... .......... 3% 196M 1s 2019-11-25T09:29:35.791 INFO:teuthology.orchestra.run.smithi153.stderr: 2700K .......... .......... .......... .......... .......... 3% 184M 1s 2019-11-25T09:29:35.791 INFO:teuthology.orchestra.run.smithi153.stderr: 2750K .......... .......... .......... .......... .......... 3% 148M 1s 2019-11-25T09:29:35.791 INFO:teuthology.orchestra.run.smithi153.stderr: 2800K .......... .......... .......... .......... .......... 3% 176M 1s 2019-11-25T09:29:35.792 INFO:teuthology.orchestra.run.smithi153.stderr: 2850K .......... .......... .......... .......... .......... 3% 175M 1s 2019-11-25T09:29:35.792 INFO:teuthology.orchestra.run.smithi153.stderr: 2900K .......... .......... .......... .......... .......... 4% 169M 1s 2019-11-25T09:29:35.792 INFO:teuthology.orchestra.run.smithi153.stderr: 2950K .......... .......... .......... .......... .......... 4% 146M 1s 2019-11-25T09:29:35.792 INFO:teuthology.orchestra.run.smithi153.stderr: 3000K .......... .......... .......... .......... .......... 4% 186M 1s 2019-11-25T09:29:35.792 INFO:teuthology.orchestra.run.smithi153.stderr: 3050K .......... .......... .......... .......... .......... 4% 182M 1s 2019-11-25T09:29:35.792 INFO:teuthology.orchestra.run.smithi153.stderr: 3100K .......... .......... .......... .......... .......... 4% 176M 1s 2019-11-25T09:29:35.792 INFO:teuthology.orchestra.run.smithi153.stderr: 3150K .......... .......... .......... .......... .......... 4% 166M 1s 2019-11-25T09:29:35.793 INFO:teuthology.orchestra.run.smithi153.stderr: 3200K .......... .......... .......... .......... .......... 4% 188M 1s 2019-11-25T09:29:35.793 INFO:teuthology.orchestra.run.smithi153.stderr: 3250K .......... .......... .......... .......... .......... 4% 184M 1s 2019-11-25T09:29:35.793 INFO:teuthology.orchestra.run.smithi153.stderr: 3300K .......... .......... .......... .......... .......... 4% 208M 1s 2019-11-25T09:29:35.793 INFO:teuthology.orchestra.run.smithi153.stderr: 3350K .......... .......... .......... .......... .......... 4% 178M 1s 2019-11-25T09:29:35.793 INFO:teuthology.orchestra.run.smithi153.stderr: 3400K .......... .......... .......... .......... .......... 4% 202M 1s 2019-11-25T09:29:35.793 INFO:teuthology.orchestra.run.smithi153.stderr: 3450K .......... .......... .......... .......... .......... 4% 212M 1s 2019-11-25T09:29:35.793 INFO:teuthology.orchestra.run.smithi153.stderr: 3500K .......... .......... .......... .......... .......... 4% 203M 1s 2019-11-25T09:29:35.794 INFO:teuthology.orchestra.run.smithi153.stderr: 3550K .......... .......... .......... .......... .......... 4% 161M 1s 2019-11-25T09:29:35.794 INFO:teuthology.orchestra.run.smithi153.stderr: 3600K .......... .......... .......... .......... .......... 4% 206M 1s 2019-11-25T09:29:35.794 INFO:teuthology.orchestra.run.smithi153.stderr: 3650K .......... .......... .......... .......... .......... 5% 218M 1s 2019-11-25T09:29:35.794 INFO:teuthology.orchestra.run.smithi153.stderr: 3700K .......... .......... .......... .......... .......... 5% 207M 1s 2019-11-25T09:29:35.794 INFO:teuthology.orchestra.run.smithi153.stderr: 3750K .......... .......... .......... .......... .......... 5% 168M 1s 2019-11-25T09:29:35.794 INFO:teuthology.orchestra.run.smithi153.stderr: 3800K .......... .......... .......... .......... .......... 5% 203M 1s 2019-11-25T09:29:35.794 INFO:teuthology.orchestra.run.smithi153.stderr: 3850K .......... .......... .......... .......... .......... 5% 203M 1s 2019-11-25T09:29:35.795 INFO:teuthology.orchestra.run.smithi153.stderr: 3900K .......... .......... .......... .......... .......... 5% 208M 1s 2019-11-25T09:29:35.795 INFO:teuthology.orchestra.run.smithi153.stderr: 3950K .......... .......... .......... .......... .......... 5% 155M 1s 2019-11-25T09:29:35.795 INFO:teuthology.orchestra.run.smithi153.stderr: 4000K .......... .......... .......... .......... .......... 5% 209M 1s 2019-11-25T09:29:35.795 INFO:teuthology.orchestra.run.smithi153.stderr: 4050K .......... .......... .......... .......... .......... 5% 146M 1s 2019-11-25T09:29:35.795 INFO:teuthology.orchestra.run.smithi153.stderr: 4100K .......... .......... .......... .......... .......... 5% 132M 1s 2019-11-25T09:29:35.795 INFO:teuthology.orchestra.run.smithi153.stderr: 4150K .......... .......... .......... .......... .......... 5% 173M 1s 2019-11-25T09:29:35.795 INFO:teuthology.orchestra.run.smithi153.stderr: 4200K .......... .......... .......... .......... .......... 5% 210M 1s 2019-11-25T09:29:35.795 INFO:teuthology.orchestra.run.smithi153.stderr: 4250K .......... .......... .......... .......... .......... 5% 214M 1s 2019-11-25T09:29:35.796 INFO:teuthology.orchestra.run.smithi153.stderr: 4300K .......... .......... .......... .......... .......... 5% 148M 1s 2019-11-25T09:29:35.796 INFO:teuthology.orchestra.run.smithi153.stderr: 4350K .......... .......... .......... .......... .......... 5% 182M 1s 2019-11-25T09:29:35.796 INFO:teuthology.orchestra.run.smithi153.stderr: 4400K .......... .......... .......... .......... .......... 6% 204M 1s 2019-11-25T09:29:35.796 INFO:teuthology.orchestra.run.smithi153.stderr: 4450K .......... .......... .......... .......... .......... 6% 202M 1s 2019-11-25T09:29:35.796 INFO:teuthology.orchestra.run.smithi153.stderr: 4500K .......... .......... .......... .......... .......... 6% 198M 1s 2019-11-25T09:29:35.796 INFO:teuthology.orchestra.run.smithi153.stderr: 4550K .......... .......... .......... .......... .......... 6% 173M 1s 2019-11-25T09:29:35.796 INFO:teuthology.orchestra.run.smithi153.stderr: 4600K .......... .......... .......... .......... .......... 6% 204M 1s 2019-11-25T09:29:35.797 INFO:teuthology.orchestra.run.smithi153.stderr: 4650K .......... .......... .......... .......... .......... 6% 180M 1s 2019-11-25T09:29:35.797 INFO:teuthology.orchestra.run.smithi153.stderr: 4700K .......... .......... .......... .......... .......... 6% 187M 1s 2019-11-25T09:29:35.797 INFO:teuthology.orchestra.run.smithi153.stderr: 4750K .......... .......... .......... .......... .......... 6% 172M 1s 2019-11-25T09:29:35.797 INFO:teuthology.orchestra.run.smithi153.stderr: 4800K .......... .......... .......... .......... .......... 6% 172M 1s 2019-11-25T09:29:35.797 INFO:teuthology.orchestra.run.smithi153.stderr: 4850K .......... .......... .......... .......... .......... 6% 173M 1s 2019-11-25T09:29:35.797 INFO:teuthology.orchestra.run.smithi153.stderr: 4900K .......... .......... .......... .......... .......... 6% 186M 1s 2019-11-25T09:29:35.797 INFO:teuthology.orchestra.run.smithi153.stderr: 4950K .......... .......... .......... .......... .......... 6% 161M 1s 2019-11-25T09:29:35.798 INFO:teuthology.orchestra.run.smithi153.stderr: 5000K .......... .......... .......... .......... .......... 6% 213M 1s 2019-11-25T09:29:35.798 INFO:teuthology.orchestra.run.smithi153.stderr: 5050K .......... .......... .......... .......... .......... 6% 188M 1s 2019-11-25T09:29:35.798 INFO:teuthology.orchestra.run.smithi153.stderr: 5100K .......... .......... .......... .......... .......... 7% 151M 1s 2019-11-25T09:29:35.798 INFO:teuthology.orchestra.run.smithi153.stderr: 5150K .......... .......... .......... .......... .......... 7% 132M 1s 2019-11-25T09:29:35.798 INFO:teuthology.orchestra.run.smithi153.stderr: 5200K .......... .......... .......... .......... .......... 7% 151M 1s 2019-11-25T09:29:35.798 INFO:teuthology.orchestra.run.smithi153.stderr: 5250K .......... .......... .......... .......... .......... 7% 152M 1s 2019-11-25T09:29:35.798 INFO:teuthology.orchestra.run.smithi153.stderr: 5300K .......... .......... .......... .......... .......... 7% 145M 1s 2019-11-25T09:29:35.799 INFO:teuthology.orchestra.run.smithi153.stderr: 5350K .......... .......... .......... .......... .......... 7% 121M 1s 2019-11-25T09:29:35.799 INFO:teuthology.orchestra.run.smithi153.stderr: 5400K .......... .......... .......... .......... .......... 7% 155M 1s 2019-11-25T09:29:35.799 INFO:teuthology.orchestra.run.smithi153.stderr: 5450K .......... .......... .......... .......... .......... 7% 156M 1s 2019-11-25T09:29:35.799 INFO:teuthology.orchestra.run.smithi153.stderr: 5500K .......... .......... .......... .......... .......... 7% 155M 1s 2019-11-25T09:29:35.799 INFO:teuthology.orchestra.run.smithi153.stderr: 5550K .......... .......... .......... .......... .......... 7% 139M 1s 2019-11-25T09:29:35.799 INFO:teuthology.orchestra.run.smithi153.stderr: 5600K .......... .......... .......... .......... .......... 7% 156M 1s 2019-11-25T09:29:35.799 INFO:teuthology.orchestra.run.smithi153.stderr: 5650K .......... .......... .......... .......... .......... 7% 155M 1s 2019-11-25T09:29:35.800 INFO:teuthology.orchestra.run.smithi153.stderr: 5700K .......... .......... .......... .......... .......... 7% 151M 1s 2019-11-25T09:29:35.800 INFO:teuthology.orchestra.run.smithi153.stderr: 5750K .......... .......... .......... .......... .......... 7% 127M 1s 2019-11-25T09:29:35.800 INFO:teuthology.orchestra.run.smithi153.stderr: 5800K .......... .......... .......... .......... .......... 7% 147M 1s 2019-11-25T09:29:35.800 INFO:teuthology.orchestra.run.smithi153.stderr: 5850K .......... .......... .......... .......... .......... 8% 152M 1s 2019-11-25T09:29:35.800 INFO:teuthology.orchestra.run.smithi153.stderr: 5900K .......... .......... .......... .......... .......... 8% 156M 1s 2019-11-25T09:29:35.800 INFO:teuthology.orchestra.run.smithi153.stderr: 5950K .......... .......... .......... .......... .......... 8% 132M 1s 2019-11-25T09:29:35.800 INFO:teuthology.orchestra.run.smithi153.stderr: 6000K .......... .......... .......... .......... .......... 8% 158M 1s 2019-11-25T09:29:35.801 INFO:teuthology.orchestra.run.smithi153.stderr: 6050K .......... .......... .......... .......... .......... 8% 148M 1s 2019-11-25T09:29:35.801 INFO:teuthology.orchestra.run.smithi153.stderr: 6100K .......... .......... .......... .......... .......... 8% 154M 1s 2019-11-25T09:29:35.801 INFO:teuthology.orchestra.run.smithi153.stderr: 6150K .......... .......... .......... .......... .......... 8% 122M 1s 2019-11-25T09:29:35.801 INFO:teuthology.orchestra.run.smithi153.stderr: 6200K .......... .......... .......... .......... .......... 8% 144M 1s 2019-11-25T09:29:35.801 INFO:teuthology.orchestra.run.smithi153.stderr: 6250K .......... .......... .......... .......... .......... 8% 148M 1s 2019-11-25T09:29:35.801 INFO:teuthology.orchestra.run.smithi153.stderr: 6300K .......... .......... .......... .......... .......... 8% 150M 1s 2019-11-25T09:29:35.801 INFO:teuthology.orchestra.run.smithi153.stderr: 6350K .......... .......... .......... .......... .......... 8% 140M 1s 2019-11-25T09:29:35.802 INFO:teuthology.orchestra.run.smithi153.stderr: 6400K .......... .......... .......... .......... .......... 8% 149M 1s 2019-11-25T09:29:35.802 INFO:teuthology.orchestra.run.smithi153.stderr: 6450K .......... .......... .......... .......... .......... 8% 155M 1s 2019-11-25T09:29:35.802 INFO:teuthology.orchestra.run.smithi153.stderr: 6500K .......... .......... .......... .......... .......... 8% 149M 1s 2019-11-25T09:29:35.802 INFO:teuthology.orchestra.run.smithi153.stderr: 6550K .......... .......... .......... .......... .......... 8% 132M 1s 2019-11-25T09:29:35.802 INFO:teuthology.orchestra.run.smithi153.stderr: 6600K .......... .......... .......... .......... .......... 9% 168M 1s 2019-11-25T09:29:35.802 INFO:teuthology.orchestra.run.smithi153.stderr: 6650K .......... .......... .......... .......... .......... 9% 152M 1s 2019-11-25T09:29:35.802 INFO:teuthology.orchestra.run.smithi153.stderr: 6700K .......... .......... .......... .......... .......... 9% 149M 1s 2019-11-25T09:29:35.803 INFO:teuthology.orchestra.run.smithi153.stderr: 6750K .......... .......... .......... .......... .......... 9% 135M 1s 2019-11-25T09:29:35.803 INFO:teuthology.orchestra.run.smithi153.stderr: 6800K .......... .......... .......... .......... .......... 9% 151M 1s 2019-11-25T09:29:35.803 INFO:teuthology.orchestra.run.smithi153.stderr: 6850K .......... .......... .......... .......... .......... 9% 152M 1s 2019-11-25T09:29:35.803 INFO:teuthology.orchestra.run.smithi153.stderr: 6900K .......... .......... .......... .......... .......... 9% 151M 1s 2019-11-25T09:29:35.803 INFO:teuthology.orchestra.run.smithi153.stderr: 6950K .......... .......... .......... .......... .......... 9% 117M 1s 2019-11-25T09:29:35.803 INFO:teuthology.orchestra.run.smithi153.stderr: 7000K .......... .......... .......... .......... .......... 9% 163M 1s 2019-11-25T09:29:35.803 INFO:teuthology.orchestra.run.smithi153.stderr: 7050K .......... .......... .......... .......... .......... 9% 152M 1s 2019-11-25T09:29:35.804 INFO:teuthology.orchestra.run.smithi153.stderr: 7100K .......... .......... .......... .......... .......... 9% 153M 1s 2019-11-25T09:29:35.804 INFO:teuthology.orchestra.run.smithi153.stderr: 7150K .......... .......... .......... .......... .......... 9% 142M 1s 2019-11-25T09:29:35.804 INFO:teuthology.orchestra.run.smithi153.stderr: 7200K .......... .......... .......... .......... .......... 9% 151M 1s 2019-11-25T09:29:35.804 INFO:teuthology.orchestra.run.smithi153.stderr: 7250K .......... .......... .......... .......... .......... 9% 142M 1s 2019-11-25T09:29:35.804 INFO:teuthology.orchestra.run.smithi153.stderr: 7300K .......... .......... .......... .......... .......... 9% 134M 1s 2019-11-25T09:29:35.804 INFO:teuthology.orchestra.run.smithi153.stderr: 7350K .......... .......... .......... .......... .......... 10% 112M 1s 2019-11-25T09:29:35.804 INFO:teuthology.orchestra.run.smithi153.stderr: 7400K .......... .......... .......... .......... .......... 10% 139M 1s 2019-11-25T09:29:35.805 INFO:teuthology.orchestra.run.smithi153.stderr: 7450K .......... .......... .......... .......... .......... 10% 165M 1s 2019-11-25T09:29:35.805 INFO:teuthology.orchestra.run.smithi153.stderr: 7500K .......... .......... .......... .......... .......... 10% 181M 1s 2019-11-25T09:29:35.805 INFO:teuthology.orchestra.run.smithi153.stderr: 7550K .......... .......... .......... .......... .......... 10% 123M 1s 2019-11-25T09:29:35.805 INFO:teuthology.orchestra.run.smithi153.stderr: 7600K .......... .......... .......... .......... .......... 10% 190M 1s 2019-11-25T09:29:35.805 INFO:teuthology.orchestra.run.smithi153.stderr: 7650K .......... .......... .......... .......... .......... 10% 186M 1s 2019-11-25T09:29:35.805 INFO:teuthology.orchestra.run.smithi153.stderr: 7700K .......... .......... .......... .......... .......... 10% 208M 1s 2019-11-25T09:29:35.805 INFO:teuthology.orchestra.run.smithi153.stderr: 7750K .......... .......... .......... .......... .......... 10% 177M 1s 2019-11-25T09:29:35.805 INFO:teuthology.orchestra.run.smithi153.stderr: 7800K .......... .......... .......... .......... .......... 10% 217M 1s 2019-11-25T09:29:35.806 INFO:teuthology.orchestra.run.smithi153.stderr: 7850K .......... .......... .......... .......... .......... 10% 229M 1s 2019-11-25T09:29:35.806 INFO:teuthology.orchestra.run.smithi153.stderr: 7900K .......... .......... .......... .......... .......... 10% 234M 1s 2019-11-25T09:29:35.806 INFO:teuthology.orchestra.run.smithi153.stderr: 7950K .......... .......... .......... .......... .......... 10% 166M 1s 2019-11-25T09:29:35.806 INFO:teuthology.orchestra.run.smithi153.stderr: 8000K .......... .......... .......... .......... .......... 10% 232M 1s 2019-11-25T09:29:35.806 INFO:teuthology.orchestra.run.smithi153.stderr: 8050K .......... .......... .......... .......... .......... 11% 197M 1s 2019-11-25T09:29:35.806 INFO:teuthology.orchestra.run.smithi153.stderr: 8100K .......... .......... .......... .......... .......... 11% 220M 1s 2019-11-25T09:29:35.806 INFO:teuthology.orchestra.run.smithi153.stderr: 8150K .......... .......... .......... .......... .......... 11% 172M 1s 2019-11-25T09:29:35.807 INFO:teuthology.orchestra.run.smithi153.stderr: 8200K .......... .......... .......... .......... .......... 11% 222M 1s 2019-11-25T09:29:35.807 INFO:teuthology.orchestra.run.smithi153.stderr: 8250K .......... .......... .......... .......... .......... 11% 229M 1s 2019-11-25T09:29:35.807 INFO:teuthology.orchestra.run.smithi153.stderr: 8300K .......... .......... .......... .......... .......... 11% 217M 1s 2019-11-25T09:29:35.807 INFO:teuthology.orchestra.run.smithi153.stderr: 8350K .......... .......... .......... .......... .......... 11% 209M 1s 2019-11-25T09:29:35.807 INFO:teuthology.orchestra.run.smithi153.stderr: 8400K .......... .......... .......... .......... .......... 11% 223M 1s 2019-11-25T09:29:35.807 INFO:teuthology.orchestra.run.smithi153.stderr: 8450K .......... .......... .......... .......... .......... 11% 234M 1s 2019-11-25T09:29:35.807 INFO:teuthology.orchestra.run.smithi153.stderr: 8500K .......... .......... .......... .......... .......... 11% 232M 1s 2019-11-25T09:29:35.808 INFO:teuthology.orchestra.run.smithi153.stderr: 8550K .......... .......... .......... .......... .......... 11% 197M 1s 2019-11-25T09:29:35.808 INFO:teuthology.orchestra.run.smithi153.stderr: 8600K .......... .......... .......... .......... .......... 11% 220M 1s 2019-11-25T09:29:35.808 INFO:teuthology.orchestra.run.smithi153.stderr: 8650K .......... .......... .......... .......... .......... 11% 214M 1s 2019-11-25T09:29:35.808 INFO:teuthology.orchestra.run.smithi153.stderr: 8700K .......... .......... .......... .......... .......... 11% 203M 1s 2019-11-25T09:29:35.808 INFO:teuthology.orchestra.run.smithi153.stderr: 8750K .......... .......... .......... .......... .......... 11% 195M 1s 2019-11-25T09:29:35.808 INFO:teuthology.orchestra.run.smithi153.stderr: 8800K .......... .......... .......... .......... .......... 12% 236M 1s 2019-11-25T09:29:35.808 INFO:teuthology.orchestra.run.smithi153.stderr: 8850K .......... .......... .......... .......... .......... 12% 199M 1s 2019-11-25T09:29:35.809 INFO:teuthology.orchestra.run.smithi153.stderr: 8900K .......... .......... .......... .......... .......... 12% 250M 1s 2019-11-25T09:29:35.809 INFO:teuthology.orchestra.run.smithi153.stderr: 8950K .......... .......... .......... .......... .......... 12% 191M 1s 2019-11-25T09:29:35.809 INFO:teuthology.orchestra.run.smithi153.stderr: 9000K .......... .......... .......... .......... .......... 12% 237M 1s 2019-11-25T09:29:35.809 INFO:teuthology.orchestra.run.smithi153.stderr: 9050K .......... .......... .......... .......... .......... 12% 217M 1s 2019-11-25T09:29:35.809 INFO:teuthology.orchestra.run.smithi153.stderr: 9100K .......... .......... .......... .......... .......... 12% 239M 1s 2019-11-25T09:29:35.809 INFO:teuthology.orchestra.run.smithi153.stderr: 9150K .......... .......... .......... .......... .......... 12% 195M 1s 2019-11-25T09:29:35.809 INFO:teuthology.orchestra.run.smithi153.stderr: 9200K .......... .......... .......... .......... .......... 12% 244M 1s 2019-11-25T09:29:35.810 INFO:teuthology.orchestra.run.smithi153.stderr: 9250K .......... .......... .......... .......... .......... 12% 215M 1s 2019-11-25T09:29:35.810 INFO:teuthology.orchestra.run.smithi153.stderr: 9300K .......... .......... .......... .......... .......... 12% 148M 1s 2019-11-25T09:29:35.810 INFO:teuthology.orchestra.run.smithi153.stderr: 9350K .......... .......... .......... .......... .......... 12% 189M 1s 2019-11-25T09:29:35.810 INFO:teuthology.orchestra.run.smithi153.stderr: 9400K .......... .......... .......... .......... .......... 12% 243M 1s 2019-11-25T09:29:35.810 INFO:teuthology.orchestra.run.smithi153.stderr: 9450K .......... .......... .......... .......... .......... 12% 228M 1s 2019-11-25T09:29:35.810 INFO:teuthology.orchestra.run.smithi153.stderr: 9500K .......... .......... .......... .......... .......... 12% 224M 1s 2019-11-25T09:29:35.810 INFO:teuthology.orchestra.run.smithi153.stderr: 9550K .......... .......... .......... .......... .......... 13% 213M 1s 2019-11-25T09:29:35.811 INFO:teuthology.orchestra.run.smithi153.stderr: 9600K .......... .......... .......... .......... .......... 13% 205M 1s 2019-11-25T09:29:35.811 INFO:teuthology.orchestra.run.smithi153.stderr: 9650K .......... .......... .......... .......... .......... 13% 192M 1s 2019-11-25T09:29:35.811 INFO:teuthology.orchestra.run.smithi153.stderr: 9700K .......... .......... .......... .......... .......... 13% 203M 1s 2019-11-25T09:29:35.811 INFO:teuthology.orchestra.run.smithi153.stderr: 9750K .......... .......... .......... .......... .......... 13% 132M 1s 2019-11-25T09:29:35.811 INFO:teuthology.orchestra.run.smithi153.stderr: 9800K .......... .......... .......... .......... .......... 13% 133M 1s 2019-11-25T09:29:35.811 INFO:teuthology.orchestra.run.smithi153.stderr: 9850K .......... .......... .......... .......... .......... 13% 118M 1s 2019-11-25T09:29:35.811 INFO:teuthology.orchestra.run.smithi153.stderr: 9900K .......... .......... .......... .......... .......... 13% 137M 1s 2019-11-25T09:29:35.812 INFO:teuthology.orchestra.run.smithi153.stderr: 9950K .......... .......... .......... .......... .......... 13% 122M 1s 2019-11-25T09:29:35.812 INFO:teuthology.orchestra.run.smithi153.stderr: 10000K .......... .......... .......... .......... .......... 13% 152M 1s 2019-11-25T09:29:35.812 INFO:teuthology.orchestra.run.smithi153.stderr: 10050K .......... .......... .......... .......... .......... 13% 146M 1s 2019-11-25T09:29:35.812 INFO:teuthology.orchestra.run.smithi153.stderr: 10100K .......... .......... .......... .......... .......... 13% 147M 1s 2019-11-25T09:29:35.812 INFO:teuthology.orchestra.run.smithi153.stderr: 10150K .......... .......... .......... .......... .......... 13% 196M 1s 2019-11-25T09:29:35.812 INFO:teuthology.orchestra.run.smithi153.stderr: 10200K .......... .......... .......... .......... .......... 13% 161M 1s 2019-11-25T09:29:35.812 INFO:teuthology.orchestra.run.smithi153.stderr: 10250K .......... .......... .......... .......... .......... 14% 161M 1s 2019-11-25T09:29:35.813 INFO:teuthology.orchestra.run.smithi153.stderr: 10300K .......... .......... .......... .......... .......... 14% 207M 1s 2019-11-25T09:29:35.813 INFO:teuthology.orchestra.run.smithi153.stderr: 10350K .......... .......... .......... .......... .......... 14% 212M 1s 2019-11-25T09:29:35.813 INFO:teuthology.orchestra.run.smithi153.stderr: 10400K .......... .......... .......... .......... .......... 14% 251M 1s 2019-11-25T09:29:35.813 INFO:teuthology.orchestra.run.smithi153.stderr: 10450K .......... .......... .......... .......... .......... 14% 237M 1s 2019-11-25T09:29:35.813 INFO:teuthology.orchestra.run.smithi153.stderr: 10500K .......... .......... .......... .......... .......... 14% 246M 1s 2019-11-25T09:29:35.813 INFO:teuthology.orchestra.run.smithi153.stderr: 10550K .......... .......... .......... .......... .......... 14% 151M 1s 2019-11-25T09:29:35.813 INFO:teuthology.orchestra.run.smithi153.stderr: 10600K .......... .......... .......... .......... .......... 14% 159M 1s 2019-11-25T09:29:35.814 INFO:teuthology.orchestra.run.smithi153.stderr: 10650K .......... .......... .......... .......... .......... 14% 186M 1s 2019-11-25T09:29:35.814 INFO:teuthology.orchestra.run.smithi153.stderr: 10700K .......... .......... .......... .......... .......... 14% 223M 1s 2019-11-25T09:29:35.814 INFO:teuthology.orchestra.run.smithi153.stderr: 10750K .......... .......... .......... .......... .......... 14% 221M 1s 2019-11-25T09:29:35.814 INFO:teuthology.orchestra.run.smithi153.stderr: 10800K .......... .......... .......... .......... .......... 14% 217M 1s 2019-11-25T09:29:35.814 INFO:teuthology.orchestra.run.smithi153.stderr: 10850K .......... .......... .......... .......... .......... 14% 248M 1s 2019-11-25T09:29:35.814 INFO:teuthology.orchestra.run.smithi153.stderr: 10900K .......... .......... .......... .......... .......... 14% 199M 1s 2019-11-25T09:29:35.814 INFO:teuthology.orchestra.run.smithi153.stderr: 10950K .......... .......... .......... .......... .......... 14% 156M 1s 2019-11-25T09:29:35.814 INFO:teuthology.orchestra.run.smithi153.stderr: 11000K .......... .......... .......... .......... .......... 15% 213M 1s 2019-11-25T09:29:35.815 INFO:teuthology.orchestra.run.smithi153.stderr: 11050K .......... .......... .......... .......... .......... 15% 238M 1s 2019-11-25T09:29:35.815 INFO:teuthology.orchestra.run.smithi153.stderr: 11100K .......... .......... .......... .......... .......... 15% 238M 1s 2019-11-25T09:29:35.815 INFO:teuthology.orchestra.run.smithi153.stderr: 11150K .......... .......... .......... .......... .......... 15% 219M 1s 2019-11-25T09:29:35.815 INFO:teuthology.orchestra.run.smithi153.stderr: 11200K .......... .......... .......... .......... .......... 15% 245M 1s 2019-11-25T09:29:35.815 INFO:teuthology.orchestra.run.smithi153.stderr: 11250K .......... .......... .......... .......... .......... 15% 261M 1s 2019-11-25T09:29:35.815 INFO:teuthology.orchestra.run.smithi153.stderr: 11300K .......... .......... .......... .......... .......... 15% 252M 1s 2019-11-25T09:29:35.815 INFO:teuthology.orchestra.run.smithi153.stderr: 11350K .......... .......... .......... .......... .......... 15% 207M 1s 2019-11-25T09:29:35.816 INFO:teuthology.orchestra.run.smithi153.stderr: 11400K .......... .......... .......... .......... .......... 15% 248M 1s 2019-11-25T09:29:35.816 INFO:teuthology.orchestra.run.smithi153.stderr: 11450K .......... .......... .......... .......... .......... 15% 263M 1s 2019-11-25T09:29:35.816 INFO:teuthology.orchestra.run.smithi153.stderr: 11500K .......... .......... .......... .......... .......... 15% 249M 1s 2019-11-25T09:29:35.816 INFO:teuthology.orchestra.run.smithi153.stderr: 11550K .......... .......... .......... .......... .......... 15% 155M 1s 2019-11-25T09:29:35.816 INFO:teuthology.orchestra.run.smithi153.stderr: 11600K .......... .......... .......... .......... .......... 15% 151M 1s 2019-11-25T09:29:35.816 INFO:teuthology.orchestra.run.smithi153.stderr: 11650K .......... .......... .......... .......... .......... 15% 156M 1s 2019-11-25T09:29:35.816 INFO:teuthology.orchestra.run.smithi153.stderr: 11700K .......... .......... .......... .......... .......... 15% 144M 1s 2019-11-25T09:29:35.817 INFO:teuthology.orchestra.run.smithi153.stderr: 11750K .......... .......... .......... .......... .......... 16% 160M 1s 2019-11-25T09:29:35.817 INFO:teuthology.orchestra.run.smithi153.stderr: 11800K .......... .......... .......... .......... .......... 16% 182M 1s 2019-11-25T09:29:35.817 INFO:teuthology.orchestra.run.smithi153.stderr: 11850K .......... .......... .......... .......... .......... 16% 147M 1s 2019-11-25T09:29:35.817 INFO:teuthology.orchestra.run.smithi153.stderr: 11900K .......... .......... .......... .......... .......... 16% 146M 1s 2019-11-25T09:29:35.817 INFO:teuthology.orchestra.run.smithi153.stderr: 11950K .......... .......... .......... .......... .......... 16% 136M 1s 2019-11-25T09:29:35.817 INFO:teuthology.orchestra.run.smithi153.stderr: 12000K .......... .......... .......... .......... .......... 16% 158M 1s 2019-11-25T09:29:35.817 INFO:teuthology.orchestra.run.smithi153.stderr: 12050K .......... .......... .......... .......... .......... 16% 145M 1s 2019-11-25T09:29:35.818 INFO:teuthology.orchestra.run.smithi153.stderr: 12100K .......... .......... .......... .......... .......... 16% 178M 1s 2019-11-25T09:29:35.818 INFO:teuthology.orchestra.run.smithi153.stderr: 12150K .......... .......... .......... .......... .......... 16% 191M 1s 2019-11-25T09:29:35.818 INFO:teuthology.orchestra.run.smithi153.stderr: 12200K .......... .......... .......... .......... .......... 16% 262M 1s 2019-11-25T09:29:35.818 INFO:teuthology.orchestra.run.smithi153.stderr: 12250K .......... .......... .......... .......... .......... 16% 238M 1s 2019-11-25T09:29:35.818 INFO:teuthology.orchestra.run.smithi153.stderr: 12300K .......... .......... .......... .......... .......... 16% 232M 1s 2019-11-25T09:29:35.818 INFO:teuthology.orchestra.run.smithi153.stderr: 12350K .......... .......... .......... .......... .......... 16% 216M 0s 2019-11-25T09:29:35.818 INFO:teuthology.orchestra.run.smithi153.stderr: 12400K .......... .......... .......... .......... .......... 16% 248M 0s 2019-11-25T09:29:35.819 INFO:teuthology.orchestra.run.smithi153.stderr: 12450K .......... .......... .......... .......... .......... 16% 231M 0s 2019-11-25T09:29:35.819 INFO:teuthology.orchestra.run.smithi153.stderr: 12500K .......... .......... .......... .......... .......... 17% 207M 0s 2019-11-25T09:29:35.819 INFO:teuthology.orchestra.run.smithi153.stderr: 12550K .......... .......... .......... .......... .......... 17% 219M 0s 2019-11-25T09:29:35.819 INFO:teuthology.orchestra.run.smithi153.stderr: 12600K .......... .......... .......... .......... .......... 17% 218M 0s 2019-11-25T09:29:35.819 INFO:teuthology.orchestra.run.smithi153.stderr: 12650K .......... .......... .......... .......... .......... 17% 214M 0s 2019-11-25T09:29:35.819 INFO:teuthology.orchestra.run.smithi153.stderr: 12700K .......... .......... .......... .......... .......... 17% 226M 0s 2019-11-25T09:29:35.819 INFO:teuthology.orchestra.run.smithi153.stderr: 12750K .......... .......... .......... .......... .......... 17% 213M 0s 2019-11-25T09:29:35.820 INFO:teuthology.orchestra.run.smithi153.stderr: 12800K .......... .......... .......... .......... .......... 17% 212M 0s 2019-11-25T09:29:35.820 INFO:teuthology.orchestra.run.smithi153.stderr: 12850K .......... .......... .......... .......... .......... 17% 239M 0s 2019-11-25T09:29:35.820 INFO:teuthology.orchestra.run.smithi153.stderr: 12900K .......... .......... .......... .......... .......... 17% 250M 0s 2019-11-25T09:29:35.820 INFO:teuthology.orchestra.run.smithi153.stderr: 12950K .......... .......... .......... .......... .......... 17% 163M 0s 2019-11-25T09:29:35.820 INFO:teuthology.orchestra.run.smithi153.stderr: 13000K .......... .......... .......... .......... .......... 17% 265M 0s 2019-11-25T09:29:35.820 INFO:teuthology.orchestra.run.smithi153.stderr: 13050K .......... .......... .......... .......... .......... 17% 233M 0s 2019-11-25T09:29:35.820 INFO:teuthology.orchestra.run.smithi153.stderr: 13100K .......... .......... .......... .......... .......... 17% 227M 0s 2019-11-25T09:29:35.821 INFO:teuthology.orchestra.run.smithi153.stderr: 13150K .......... .......... .......... .......... .......... 17% 216M 0s 2019-11-25T09:29:35.821 INFO:teuthology.orchestra.run.smithi153.stderr: 13200K .......... .......... .......... .......... .......... 18% 232M 0s 2019-11-25T09:29:35.821 INFO:teuthology.orchestra.run.smithi153.stderr: 13250K .......... .......... .......... .......... .......... 18% 246M 0s 2019-11-25T09:29:35.821 INFO:teuthology.orchestra.run.smithi153.stderr: 13300K .......... .......... .......... .......... .......... 18% 263M 0s 2019-11-25T09:29:35.821 INFO:teuthology.orchestra.run.smithi153.stderr: 13350K .......... .......... .......... .......... .......... 18% 208M 0s 2019-11-25T09:29:35.821 INFO:teuthology.orchestra.run.smithi153.stderr: 13400K .......... .......... .......... .......... .......... 18% 231M 0s 2019-11-25T09:29:35.821 INFO:teuthology.orchestra.run.smithi153.stderr: 13450K .......... .......... .......... .......... .......... 18% 238M 0s 2019-11-25T09:29:35.822 INFO:teuthology.orchestra.run.smithi153.stderr: 13500K .......... .......... .......... .......... .......... 18% 249M 0s 2019-11-25T09:29:35.822 INFO:teuthology.orchestra.run.smithi153.stderr: 13550K .......... .......... .......... .......... .......... 18% 192M 0s 2019-11-25T09:29:35.822 INFO:teuthology.orchestra.run.smithi153.stderr: 13600K .......... .......... .......... .......... .......... 18% 195M 0s 2019-11-25T09:29:35.822 INFO:teuthology.orchestra.run.smithi153.stderr: 13650K .......... .......... .......... .......... .......... 18% 260M 0s 2019-11-25T09:29:35.822 INFO:teuthology.orchestra.run.smithi153.stderr: 13700K .......... .......... .......... .......... .......... 18% 270M 0s 2019-11-25T09:29:35.822 INFO:teuthology.orchestra.run.smithi153.stderr: 13750K .......... .......... .......... .......... .......... 18% 206M 0s 2019-11-25T09:29:35.822 INFO:teuthology.orchestra.run.smithi153.stderr: 13800K .......... .......... .......... .......... .......... 18% 226M 0s 2019-11-25T09:29:35.823 INFO:teuthology.orchestra.run.smithi153.stderr: 13850K .......... .......... .......... .......... .......... 18% 251M 0s 2019-11-25T09:29:35.823 INFO:teuthology.orchestra.run.smithi153.stderr: 13900K .......... .......... .......... .......... .......... 18% 247M 0s 2019-11-25T09:29:35.823 INFO:teuthology.orchestra.run.smithi153.stderr: 13950K .......... .......... .......... .......... .......... 19% 211M 0s 2019-11-25T09:29:35.823 INFO:teuthology.orchestra.run.smithi153.stderr: 14000K .......... .......... .......... .......... .......... 19% 229M 0s 2019-11-25T09:29:35.823 INFO:teuthology.orchestra.run.smithi153.stderr: 14050K .......... .......... .......... .......... .......... 19% 213M 0s 2019-11-25T09:29:35.823 INFO:teuthology.orchestra.run.smithi153.stderr: 14100K .......... .......... .......... .......... .......... 19% 211M 0s 2019-11-25T09:29:35.823 INFO:teuthology.orchestra.run.smithi153.stderr: 14150K .......... .......... .......... .......... .......... 19% 185M 0s 2019-11-25T09:29:35.824 INFO:teuthology.orchestra.run.smithi153.stderr: 14200K .......... .......... .......... .......... .......... 19% 215M 0s 2019-11-25T09:29:35.824 INFO:teuthology.orchestra.run.smithi153.stderr: 14250K .......... .......... .......... .......... .......... 19% 258M 0s 2019-11-25T09:29:35.824 INFO:teuthology.orchestra.run.smithi153.stderr: 14300K .......... .......... .......... .......... .......... 19% 244M 0s 2019-11-25T09:29:35.824 INFO:teuthology.orchestra.run.smithi153.stderr: 14350K .......... .......... .......... .......... .......... 19% 179M 0s 2019-11-25T09:29:35.824 INFO:teuthology.orchestra.run.smithi153.stderr: 14400K .......... .......... .......... .......... .......... 19% 249M 0s 2019-11-25T09:29:35.824 INFO:teuthology.orchestra.run.smithi153.stderr: 14450K .......... .......... .......... .......... .......... 19% 230M 0s 2019-11-25T09:29:35.824 INFO:teuthology.orchestra.run.smithi153.stderr: 14500K .......... .......... .......... .......... .......... 19% 236M 0s 2019-11-25T09:29:35.825 INFO:teuthology.orchestra.run.smithi153.stderr: 14550K .......... .......... .......... .......... .......... 19% 178M 0s 2019-11-25T09:29:35.825 INFO:teuthology.orchestra.run.smithi153.stderr: 14600K .......... .......... .......... .......... .......... 19% 270M 0s 2019-11-25T09:29:35.825 INFO:teuthology.orchestra.run.smithi153.stderr: 14650K .......... .......... .......... .......... .......... 19% 266M 0s 2019-11-25T09:29:35.825 INFO:teuthology.orchestra.run.smithi153.stderr: 14700K .......... .......... .......... .......... .......... 20% 251M 0s 2019-11-25T09:29:35.825 INFO:teuthology.orchestra.run.smithi153.stderr: 14750K .......... .......... .......... .......... .......... 20% 227M 0s 2019-11-25T09:29:35.825 INFO:teuthology.orchestra.run.smithi153.stderr: 14800K .......... .......... .......... .......... .......... 20% 237M 0s 2019-11-25T09:29:35.825 INFO:teuthology.orchestra.run.smithi153.stderr: 14850K .......... .......... .......... .......... .......... 20% 266M 0s 2019-11-25T09:29:35.825 INFO:teuthology.orchestra.run.smithi153.stderr: 14900K .......... .......... .......... .......... .......... 20% 248M 0s 2019-11-25T09:29:35.826 INFO:teuthology.orchestra.run.smithi153.stderr: 14950K .......... .......... .......... .......... .......... 20% 209M 0s 2019-11-25T09:29:35.826 INFO:teuthology.orchestra.run.smithi153.stderr: 15000K .......... .......... .......... .......... .......... 20% 240M 0s 2019-11-25T09:29:35.826 INFO:teuthology.orchestra.run.smithi153.stderr: 15050K .......... .......... .......... .......... .......... 20% 266M 0s 2019-11-25T09:29:35.826 INFO:teuthology.orchestra.run.smithi153.stderr: 15100K .......... .......... .......... .......... .......... 20% 254M 0s 2019-11-25T09:29:35.826 INFO:teuthology.orchestra.run.smithi153.stderr: 15150K .......... .......... .......... .......... .......... 20% 229M 0s 2019-11-25T09:29:35.826 INFO:teuthology.orchestra.run.smithi153.stderr: 15200K .......... .......... .......... .......... .......... 20% 263M 0s 2019-11-25T09:29:35.826 INFO:teuthology.orchestra.run.smithi153.stderr: 15250K .......... .......... .......... .......... .......... 20% 255M 0s 2019-11-25T09:29:35.827 INFO:teuthology.orchestra.run.smithi153.stderr: 15300K .......... .......... .......... .......... .......... 20% 261M 0s 2019-11-25T09:29:35.827 INFO:teuthology.orchestra.run.smithi153.stderr: 15350K .......... .......... .......... .......... .......... 20% 217M 0s 2019-11-25T09:29:35.827 INFO:teuthology.orchestra.run.smithi153.stderr: 15400K .......... .......... .......... .......... .......... 21% 249M 0s 2019-11-25T09:29:35.827 INFO:teuthology.orchestra.run.smithi153.stderr: 15450K .......... .......... .......... .......... .......... 21% 256M 0s 2019-11-25T09:29:35.827 INFO:teuthology.orchestra.run.smithi153.stderr: 15500K .......... .......... .......... .......... .......... 21% 260M 0s 2019-11-25T09:29:35.827 INFO:teuthology.orchestra.run.smithi153.stderr: 15550K .......... .......... .......... .......... .......... 21% 219M 0s 2019-11-25T09:29:35.827 INFO:teuthology.orchestra.run.smithi153.stderr: 15600K .......... .......... .......... .......... .......... 21% 217M 0s 2019-11-25T09:29:35.828 INFO:teuthology.orchestra.run.smithi153.stderr: 15650K .......... .......... .......... .......... .......... 21% 249M 0s 2019-11-25T09:29:35.828 INFO:teuthology.orchestra.run.smithi153.stderr: 15700K .......... .......... .......... .......... .......... 21% 248M 0s 2019-11-25T09:29:35.828 INFO:teuthology.orchestra.run.smithi153.stderr: 15750K .......... .......... .......... .......... .......... 21% 207M 0s 2019-11-25T09:29:35.828 INFO:teuthology.orchestra.run.smithi153.stderr: 15800K .......... .......... .......... .......... .......... 21% 242M 0s 2019-11-25T09:29:35.828 INFO:teuthology.orchestra.run.smithi153.stderr: 15850K .......... .......... .......... .......... .......... 21% 260M 0s 2019-11-25T09:29:35.828 INFO:teuthology.orchestra.run.smithi153.stderr: 15900K .......... .......... .......... .......... .......... 21% 262M 0s 2019-11-25T09:29:35.828 INFO:teuthology.orchestra.run.smithi153.stderr: 15950K .......... .......... .......... .......... .......... 21% 222M 0s 2019-11-25T09:29:35.829 INFO:teuthology.orchestra.run.smithi153.stderr: 16000K .......... .......... .......... .......... .......... 21% 254M 0s 2019-11-25T09:29:35.829 INFO:teuthology.orchestra.run.smithi153.stderr: 16050K .......... .......... .......... .......... .......... 21% 246M 0s 2019-11-25T09:29:35.829 INFO:teuthology.orchestra.run.smithi153.stderr: 16100K .......... .......... .......... .......... .......... 21% 230M 0s 2019-11-25T09:29:35.829 INFO:teuthology.orchestra.run.smithi153.stderr: 16150K .......... .......... .......... .......... .......... 22% 210M 0s 2019-11-25T09:29:35.829 INFO:teuthology.orchestra.run.smithi153.stderr: 16200K .......... .......... .......... .......... .......... 22% 279M 0s 2019-11-25T09:29:35.829 INFO:teuthology.orchestra.run.smithi153.stderr: 16250K .......... .......... .......... .......... .......... 22% 248M 0s 2019-11-25T09:29:35.829 INFO:teuthology.orchestra.run.smithi153.stderr: 16300K .......... .......... .......... .......... .......... 22% 241M 0s 2019-11-25T09:29:35.830 INFO:teuthology.orchestra.run.smithi153.stderr: 16350K .......... .......... .......... .......... .......... 22% 227M 0s 2019-11-25T09:29:35.830 INFO:teuthology.orchestra.run.smithi153.stderr: 16400K .......... .......... .......... .......... .......... 22% 261M 0s 2019-11-25T09:29:35.830 INFO:teuthology.orchestra.run.smithi153.stderr: 16450K .......... .......... .......... .......... .......... 22% 256M 0s 2019-11-25T09:29:35.830 INFO:teuthology.orchestra.run.smithi153.stderr: 16500K .......... .......... .......... .......... .......... 22% 259M 0s 2019-11-25T09:29:35.830 INFO:teuthology.orchestra.run.smithi153.stderr: 16550K .......... .......... .......... .......... .......... 22% 204M 0s 2019-11-25T09:29:35.830 INFO:teuthology.orchestra.run.smithi153.stderr: 16600K .......... .......... .......... .......... .......... 22% 257M 0s 2019-11-25T09:29:35.830 INFO:teuthology.orchestra.run.smithi153.stderr: 16650K .......... .......... .......... .......... .......... 22% 253M 0s 2019-11-25T09:29:35.830 INFO:teuthology.orchestra.run.smithi153.stderr: 16700K .......... .......... .......... .......... .......... 22% 274M 0s 2019-11-25T09:29:35.831 INFO:teuthology.orchestra.run.smithi153.stderr: 16750K .......... .......... .......... .......... .......... 22% 229M 0s 2019-11-25T09:29:35.831 INFO:teuthology.orchestra.run.smithi153.stderr: 16800K .......... .......... .......... .......... .......... 22% 201M 0s 2019-11-25T09:29:35.831 INFO:teuthology.orchestra.run.smithi153.stderr: 16850K .......... .......... .......... .......... .......... 22% 218M 0s 2019-11-25T09:29:35.831 INFO:teuthology.orchestra.run.smithi153.stderr: 16900K .......... .......... .......... .......... .......... 23% 281M 0s 2019-11-25T09:29:35.831 INFO:teuthology.orchestra.run.smithi153.stderr: 16950K .......... .......... .......... .......... .......... 23% 219M 0s 2019-11-25T09:29:35.831 INFO:teuthology.orchestra.run.smithi153.stderr: 17000K .......... .......... .......... .......... .......... 23% 243M 0s 2019-11-25T09:29:35.832 INFO:teuthology.orchestra.run.smithi153.stderr: 17050K .......... .......... .......... .......... .......... 23% 270M 0s 2019-11-25T09:29:35.832 INFO:teuthology.orchestra.run.smithi153.stderr: 17100K .......... .......... .......... .......... .......... 23% 264M 0s 2019-11-25T09:29:35.832 INFO:teuthology.orchestra.run.smithi153.stderr: 17150K .......... .......... .......... .......... .......... 23% 245M 0s 2019-11-25T09:29:35.832 INFO:teuthology.orchestra.run.smithi153.stderr: 17200K .......... .......... .......... .......... .......... 23% 278M 0s 2019-11-25T09:29:35.832 INFO:teuthology.orchestra.run.smithi153.stderr: 17250K .......... .......... .......... .......... .......... 23% 270M 0s 2019-11-25T09:29:35.832 INFO:teuthology.orchestra.run.smithi153.stderr: 17300K .......... .......... .......... .......... .......... 23% 248M 0s 2019-11-25T09:29:35.832 INFO:teuthology.orchestra.run.smithi153.stderr: 17350K .......... .......... .......... .......... .......... 23% 233M 0s 2019-11-25T09:29:35.833 INFO:teuthology.orchestra.run.smithi153.stderr: 17400K .......... .......... .......... .......... .......... 23% 266M 0s 2019-11-25T09:29:35.833 INFO:teuthology.orchestra.run.smithi153.stderr: 17450K .......... .......... .......... .......... .......... 23% 268M 0s 2019-11-25T09:29:35.833 INFO:teuthology.orchestra.run.smithi153.stderr: 17500K .......... .......... .......... .......... .......... 23% 264M 0s 2019-11-25T09:29:35.833 INFO:teuthology.orchestra.run.smithi153.stderr: 17550K .......... .......... .......... .......... .......... 23% 231M 0s 2019-11-25T09:29:35.833 INFO:teuthology.orchestra.run.smithi153.stderr: 17600K .......... .......... .......... .......... .......... 23% 279M 0s 2019-11-25T09:29:35.833 INFO:teuthology.orchestra.run.smithi153.stderr: 17650K .......... .......... .......... .......... .......... 24% 275M 0s 2019-11-25T09:29:35.833 INFO:teuthology.orchestra.run.smithi153.stderr: 17700K .......... .......... .......... .......... .......... 24% 276M 0s 2019-11-25T09:29:35.833 INFO:teuthology.orchestra.run.smithi153.stderr: 17750K .......... .......... .......... .......... .......... 24% 201M 0s 2019-11-25T09:29:35.834 INFO:teuthology.orchestra.run.smithi153.stderr: 17800K .......... .......... .......... .......... .......... 24% 230M 0s 2019-11-25T09:29:35.834 INFO:teuthology.orchestra.run.smithi153.stderr: 17850K .......... .......... .......... .......... .......... 24% 260M 0s 2019-11-25T09:29:35.834 INFO:teuthology.orchestra.run.smithi153.stderr: 17900K .......... .......... .......... .......... .......... 24% 244M 0s 2019-11-25T09:29:35.834 INFO:teuthology.orchestra.run.smithi153.stderr: 17950K .......... .......... .......... .......... .......... 24% 225M 0s 2019-11-25T09:29:35.834 INFO:teuthology.orchestra.run.smithi153.stderr: 18000K .......... .......... .......... .......... .......... 24% 272M 0s 2019-11-25T09:29:35.834 INFO:teuthology.orchestra.run.smithi153.stderr: 18050K .......... .......... .......... .......... .......... 24% 227M 0s 2019-11-25T09:29:35.834 INFO:teuthology.orchestra.run.smithi153.stderr: 18100K .......... .......... .......... .......... .......... 24% 275M 0s 2019-11-25T09:29:35.835 INFO:teuthology.orchestra.run.smithi153.stderr: 18150K .......... .......... .......... .......... .......... 24% 200M 0s 2019-11-25T09:29:35.835 INFO:teuthology.orchestra.run.smithi153.stderr: 18200K .......... .......... .......... .......... .......... 24% 256M 0s 2019-11-25T09:29:35.835 INFO:teuthology.orchestra.run.smithi153.stderr: 18250K .......... .......... .......... .......... .......... 24% 277M 0s 2019-11-25T09:29:35.835 INFO:teuthology.orchestra.run.smithi153.stderr: 18300K .......... .......... .......... .......... .......... 24% 248M 0s 2019-11-25T09:29:35.835 INFO:teuthology.orchestra.run.smithi153.stderr: 18350K .......... .......... .......... .......... .......... 25% 234M 0s 2019-11-25T09:29:35.835 INFO:teuthology.orchestra.run.smithi153.stderr: 18400K .......... .......... .......... .......... .......... 25% 271M 0s 2019-11-25T09:29:35.835 INFO:teuthology.orchestra.run.smithi153.stderr: 18450K .......... .......... .......... .......... .......... 25% 286M 0s 2019-11-25T09:29:35.836 INFO:teuthology.orchestra.run.smithi153.stderr: 18500K .......... .......... .......... .......... .......... 25% 249M 0s 2019-11-25T09:29:35.836 INFO:teuthology.orchestra.run.smithi153.stderr: 18550K .......... .......... .......... .......... .......... 25% 203M 0s 2019-11-25T09:29:35.836 INFO:teuthology.orchestra.run.smithi153.stderr: 18600K .......... .......... .......... .......... .......... 25% 230M 0s 2019-11-25T09:29:35.836 INFO:teuthology.orchestra.run.smithi153.stderr: 18650K .......... .......... .......... .......... .......... 25% 229M 0s 2019-11-25T09:29:35.836 INFO:teuthology.orchestra.run.smithi153.stderr: 18700K .......... .......... .......... .......... .......... 25% 271M 0s 2019-11-25T09:29:35.836 INFO:teuthology.orchestra.run.smithi153.stderr: 18750K .......... .......... .......... .......... .......... 25% 231M 0s 2019-11-25T09:29:35.836 INFO:teuthology.orchestra.run.smithi153.stderr: 18800K .......... .......... .......... .......... .......... 25% 263M 0s 2019-11-25T09:29:35.837 INFO:teuthology.orchestra.run.smithi153.stderr: 18850K .......... .......... .......... .......... .......... 25% 299M 0s 2019-11-25T09:29:35.837 INFO:teuthology.orchestra.run.smithi153.stderr: 18900K .......... .......... .......... .......... .......... 25% 259M 0s 2019-11-25T09:29:35.837 INFO:teuthology.orchestra.run.smithi153.stderr: 18950K .......... .......... .......... .......... .......... 25% 239M 0s 2019-11-25T09:29:35.837 INFO:teuthology.orchestra.run.smithi153.stderr: 19000K .......... .......... .......... .......... .......... 25% 274M 0s 2019-11-25T09:29:35.837 INFO:teuthology.orchestra.run.smithi153.stderr: 19050K .......... .......... .......... .......... .......... 25% 270M 0s 2019-11-25T09:29:35.837 INFO:teuthology.orchestra.run.smithi153.stderr: 19100K .......... .......... .......... .......... .......... 26% 284M 0s 2019-11-25T09:29:35.837 INFO:teuthology.orchestra.run.smithi153.stderr: 19150K .......... .......... .......... .......... .......... 26% 254M 0s 2019-11-25T09:29:35.838 INFO:teuthology.orchestra.run.smithi153.stderr: 19200K .......... .......... .......... .......... .......... 26% 268M 0s 2019-11-25T09:29:35.838 INFO:teuthology.orchestra.run.smithi153.stderr: 19250K .......... .......... .......... .......... .......... 26% 295M 0s 2019-11-25T09:29:35.838 INFO:teuthology.orchestra.run.smithi153.stderr: 19300K .......... .......... .......... .......... .......... 26% 277M 0s 2019-11-25T09:29:35.838 INFO:teuthology.orchestra.run.smithi153.stderr: 19350K .......... .......... .......... .......... .......... 26% 234M 0s 2019-11-25T09:29:35.838 INFO:teuthology.orchestra.run.smithi153.stderr: 19400K .......... .......... .......... .......... .......... 26% 295M 0s 2019-11-25T09:29:35.838 INFO:teuthology.orchestra.run.smithi153.stderr: 19450K .......... .......... .......... .......... .......... 26% 280M 0s 2019-11-25T09:29:35.838 INFO:teuthology.orchestra.run.smithi153.stderr: 19500K .......... .......... .......... .......... .......... 26% 263M 0s 2019-11-25T09:29:35.839 INFO:teuthology.orchestra.run.smithi153.stderr: 19550K .......... .......... .......... .......... .......... 26% 249M 0s 2019-11-25T09:29:35.839 INFO:teuthology.orchestra.run.smithi153.stderr: 19600K .......... .......... .......... .......... .......... 26% 271M 0s 2019-11-25T09:29:35.839 INFO:teuthology.orchestra.run.smithi153.stderr: 19650K .......... .......... .......... .......... .......... 26% 258M 0s 2019-11-25T09:29:35.839 INFO:teuthology.orchestra.run.smithi153.stderr: 19700K .......... .......... .......... .......... .......... 26% 266M 0s 2019-11-25T09:29:35.839 INFO:teuthology.orchestra.run.smithi153.stderr: 19750K .......... .......... .......... .......... .......... 26% 239M 0s 2019-11-25T09:29:35.839 INFO:teuthology.orchestra.run.smithi153.stderr: 19800K .......... .......... .......... .......... .......... 26% 269M 0s 2019-11-25T09:29:35.839 INFO:teuthology.orchestra.run.smithi153.stderr: 19850K .......... .......... .......... .......... .......... 27% 283M 0s 2019-11-25T09:29:35.840 INFO:teuthology.orchestra.run.smithi153.stderr: 19900K .......... .......... .......... .......... .......... 27% 267M 0s 2019-11-25T09:29:35.840 INFO:teuthology.orchestra.run.smithi153.stderr: 19950K .......... .......... .......... .......... .......... 27% 239M 0s 2019-11-25T09:29:35.840 INFO:teuthology.orchestra.run.smithi153.stderr: 20000K .......... .......... .......... .......... .......... 27% 290M 0s 2019-11-25T09:29:35.840 INFO:teuthology.orchestra.run.smithi153.stderr: 20050K .......... .......... .......... .......... .......... 27% 293M 0s 2019-11-25T09:29:35.840 INFO:teuthology.orchestra.run.smithi153.stderr: 20100K .......... .......... .......... .......... .......... 27% 288M 0s 2019-11-25T09:29:35.840 INFO:teuthology.orchestra.run.smithi153.stderr: 20150K .......... .......... .......... .......... .......... 27% 232M 0s 2019-11-25T09:29:35.840 INFO:teuthology.orchestra.run.smithi153.stderr: 20200K .......... .......... .......... .......... .......... 27% 266M 0s 2019-11-25T09:29:35.841 INFO:teuthology.orchestra.run.smithi153.stderr: 20250K .......... .......... .......... .......... .......... 27% 257M 0s 2019-11-25T09:29:35.841 INFO:teuthology.orchestra.run.smithi153.stderr: 20300K .......... .......... .......... .......... .......... 27% 272M 0s 2019-11-25T09:29:35.841 INFO:teuthology.orchestra.run.smithi153.stderr: 20350K .......... .......... .......... .......... .......... 27% 229M 0s 2019-11-25T09:29:35.841 INFO:teuthology.orchestra.run.smithi153.stderr: 20400K .......... .......... .......... .......... .......... 27% 271M 0s 2019-11-25T09:29:35.841 INFO:teuthology.orchestra.run.smithi153.stderr: 20450K .......... .......... .......... .......... .......... 27% 245M 0s 2019-11-25T09:29:35.841 INFO:teuthology.orchestra.run.smithi153.stderr: 20500K .......... .......... .......... .......... .......... 27% 255M 0s 2019-11-25T09:29:35.841 INFO:teuthology.orchestra.run.smithi153.stderr: 20550K .......... .......... .......... .......... .......... 28% 236M 0s 2019-11-25T09:29:35.841 INFO:teuthology.orchestra.run.smithi153.stderr: 20600K .......... .......... .......... .......... .......... 28% 287M 0s 2019-11-25T09:29:35.842 INFO:teuthology.orchestra.run.smithi153.stderr: 20650K .......... .......... .......... .......... .......... 28% 285M 0s 2019-11-25T09:29:35.842 INFO:teuthology.orchestra.run.smithi153.stderr: 20700K .......... .......... .......... .......... .......... 28% 282M 0s 2019-11-25T09:29:35.842 INFO:teuthology.orchestra.run.smithi153.stderr: 20750K .......... .......... .......... .......... .......... 28% 263M 0s 2019-11-25T09:29:35.842 INFO:teuthology.orchestra.run.smithi153.stderr: 20800K .......... .......... .......... .......... .......... 28% 276M 0s 2019-11-25T09:29:35.842 INFO:teuthology.orchestra.run.smithi153.stderr: 20850K .......... .......... .......... .......... .......... 28% 291M 0s 2019-11-25T09:29:35.842 INFO:teuthology.orchestra.run.smithi153.stderr: 20900K .......... .......... .......... .......... .......... 28% 303M 0s 2019-11-25T09:29:35.842 INFO:teuthology.orchestra.run.smithi153.stderr: 20950K .......... .......... .......... .......... .......... 28% 243M 0s 2019-11-25T09:29:35.843 INFO:teuthology.orchestra.run.smithi153.stderr: 21000K .......... .......... .......... .......... .......... 28% 263M 0s 2019-11-25T09:29:35.843 INFO:teuthology.orchestra.run.smithi153.stderr: 21050K .......... .......... .......... .......... .......... 28% 273M 0s 2019-11-25T09:29:35.843 INFO:teuthology.orchestra.run.smithi153.stderr: 21100K .......... .......... .......... .......... .......... 28% 291M 0s 2019-11-25T09:29:35.843 INFO:teuthology.orchestra.run.smithi153.stderr: 21150K .......... .......... .......... .......... .......... 28% 263M 0s 2019-11-25T09:29:35.843 INFO:teuthology.orchestra.run.smithi153.stderr: 21200K .......... .......... .......... .......... .......... 28% 291M 0s 2019-11-25T09:29:35.843 INFO:teuthology.orchestra.run.smithi153.stderr: 21250K .......... .......... .......... .......... .......... 28% 267M 0s 2019-11-25T09:29:35.843 INFO:teuthology.orchestra.run.smithi153.stderr: 21300K .......... .......... .......... .......... .......... 29% 231M 0s 2019-11-25T09:29:35.844 INFO:teuthology.orchestra.run.smithi153.stderr: 21350K .......... .......... .......... .......... .......... 29% 220M 0s 2019-11-25T09:29:35.844 INFO:teuthology.orchestra.run.smithi153.stderr: 21400K .......... .......... .......... .......... .......... 29% 287M 0s 2019-11-25T09:29:35.844 INFO:teuthology.orchestra.run.smithi153.stderr: 21450K .......... .......... .......... .......... .......... 29% 286M 0s 2019-11-25T09:29:35.844 INFO:teuthology.orchestra.run.smithi153.stderr: 21500K .......... .......... .......... .......... .......... 29% 253M 0s 2019-11-25T09:29:35.844 INFO:teuthology.orchestra.run.smithi153.stderr: 21550K .......... .......... .......... .......... .......... 29% 249M 0s 2019-11-25T09:29:35.844 INFO:teuthology.orchestra.run.smithi153.stderr: 21600K .......... .......... .......... .......... .......... 29% 297M 0s 2019-11-25T09:29:35.844 INFO:teuthology.orchestra.run.smithi153.stderr: 21650K .......... .......... .......... .......... .......... 29% 268M 0s 2019-11-25T09:29:35.845 INFO:teuthology.orchestra.run.smithi153.stderr: 21700K .......... .......... .......... .......... .......... 29% 304M 0s 2019-11-25T09:29:35.845 INFO:teuthology.orchestra.run.smithi153.stderr: 21750K .......... .......... .......... .......... .......... 29% 242M 0s 2019-11-25T09:29:35.845 INFO:teuthology.orchestra.run.smithi153.stderr: 21800K .......... .......... .......... .......... .......... 29% 280M 0s 2019-11-25T09:29:35.845 INFO:teuthology.orchestra.run.smithi153.stderr: 21850K .......... .......... .......... .......... .......... 29% 300M 0s 2019-11-25T09:29:35.845 INFO:teuthology.orchestra.run.smithi153.stderr: 21900K .......... .......... .......... .......... .......... 29% 311M 0s 2019-11-25T09:29:35.845 INFO:teuthology.orchestra.run.smithi153.stderr: 21950K .......... .......... .......... .......... .......... 29% 255M 0s 2019-11-25T09:29:35.845 INFO:teuthology.orchestra.run.smithi153.stderr: 22000K .......... .......... .......... .......... .......... 29% 282M 0s 2019-11-25T09:29:35.846 INFO:teuthology.orchestra.run.smithi153.stderr: 22050K .......... .......... .......... .......... .......... 30% 283M 0s 2019-11-25T09:29:35.846 INFO:teuthology.orchestra.run.smithi153.stderr: 22100K .......... .......... .......... .......... .......... 30% 281M 0s 2019-11-25T09:29:35.846 INFO:teuthology.orchestra.run.smithi153.stderr: 22150K .......... .......... .......... .......... .......... 30% 213M 0s 2019-11-25T09:29:35.846 INFO:teuthology.orchestra.run.smithi153.stderr: 22200K .......... .......... .......... .......... .......... 30% 281M 0s 2019-11-25T09:29:35.846 INFO:teuthology.orchestra.run.smithi153.stderr: 22250K .......... .......... .......... .......... .......... 30% 287M 0s 2019-11-25T09:29:35.846 INFO:teuthology.orchestra.run.smithi153.stderr: 22300K .......... .......... .......... .......... .......... 30% 271M 0s 2019-11-25T09:29:35.846 INFO:teuthology.orchestra.run.smithi153.stderr: 22350K .......... .......... .......... .......... .......... 30% 239M 0s 2019-11-25T09:29:35.847 INFO:teuthology.orchestra.run.smithi153.stderr: 22400K .......... .......... .......... .......... .......... 30% 271M 0s 2019-11-25T09:29:35.847 INFO:teuthology.orchestra.run.smithi153.stderr: 22450K .......... .......... .......... .......... .......... 30% 310M 0s 2019-11-25T09:29:35.847 INFO:teuthology.orchestra.run.smithi153.stderr: 22500K .......... .......... .......... .......... .......... 30% 275M 0s 2019-11-25T09:29:35.847 INFO:teuthology.orchestra.run.smithi153.stderr: 22550K .......... .......... .......... .......... .......... 30% 248M 0s 2019-11-25T09:29:35.847 INFO:teuthology.orchestra.run.smithi153.stderr: 22600K .......... .......... .......... .......... .......... 30% 293M 0s 2019-11-25T09:29:35.847 INFO:teuthology.orchestra.run.smithi153.stderr: 22650K .......... .......... .......... .......... .......... 30% 286M 0s 2019-11-25T09:29:35.847 INFO:teuthology.orchestra.run.smithi153.stderr: 22700K .......... .......... .......... .......... .......... 30% 289M 0s 2019-11-25T09:29:35.848 INFO:teuthology.orchestra.run.smithi153.stderr: 22750K .......... .......... .......... .......... .......... 30% 262M 0s 2019-11-25T09:29:35.848 INFO:teuthology.orchestra.run.smithi153.stderr: 22800K .......... .......... .......... .......... .......... 31% 291M 0s 2019-11-25T09:29:35.848 INFO:teuthology.orchestra.run.smithi153.stderr: 22850K .......... .......... .......... .......... .......... 31% 316M 0s 2019-11-25T09:29:35.848 INFO:teuthology.orchestra.run.smithi153.stderr: 22900K .......... .......... .......... .......... .......... 31% 303M 0s 2019-11-25T09:29:35.848 INFO:teuthology.orchestra.run.smithi153.stderr: 22950K .......... .......... .......... .......... .......... 31% 248M 0s 2019-11-25T09:29:35.848 INFO:teuthology.orchestra.run.smithi153.stderr: 23000K .......... .......... .......... .......... .......... 31% 296M 0s 2019-11-25T09:29:35.848 INFO:teuthology.orchestra.run.smithi153.stderr: 23050K .......... .......... .......... .......... .......... 31% 295M 0s 2019-11-25T09:29:35.849 INFO:teuthology.orchestra.run.smithi153.stderr: 23100K .......... .......... .......... .......... .......... 31% 273M 0s 2019-11-25T09:29:35.849 INFO:teuthology.orchestra.run.smithi153.stderr: 23150K .......... .......... .......... .......... .......... 31% 249M 0s 2019-11-25T09:29:35.849 INFO:teuthology.orchestra.run.smithi153.stderr: 23200K .......... .......... .......... .......... .......... 31% 275M 0s 2019-11-25T09:29:35.849 INFO:teuthology.orchestra.run.smithi153.stderr: 23250K .......... .......... .......... .......... .......... 31% 314M 0s 2019-11-25T09:29:35.849 INFO:teuthology.orchestra.run.smithi153.stderr: 23300K .......... .......... .......... .......... .......... 31% 297M 0s 2019-11-25T09:29:35.849 INFO:teuthology.orchestra.run.smithi153.stderr: 23350K .......... .......... .......... .......... .......... 31% 226M 0s 2019-11-25T09:29:35.849 INFO:teuthology.orchestra.run.smithi153.stderr: 23400K .......... .......... .......... .......... .......... 31% 290M 0s 2019-11-25T09:29:35.850 INFO:teuthology.orchestra.run.smithi153.stderr: 23450K .......... .......... .......... .......... .......... 31% 303M 0s 2019-11-25T09:29:35.850 INFO:teuthology.orchestra.run.smithi153.stderr: 23500K .......... .......... .......... .......... .......... 32% 269M 0s 2019-11-25T09:29:35.850 INFO:teuthology.orchestra.run.smithi153.stderr: 23550K .......... .......... .......... .......... .......... 32% 226M 0s 2019-11-25T09:29:35.850 INFO:teuthology.orchestra.run.smithi153.stderr: 23600K .......... .......... .......... .......... .......... 32% 298M 0s 2019-11-25T09:29:35.850 INFO:teuthology.orchestra.run.smithi153.stderr: 23650K .......... .......... .......... .......... .......... 32% 290M 0s 2019-11-25T09:29:35.850 INFO:teuthology.orchestra.run.smithi153.stderr: 23700K .......... .......... .......... .......... .......... 32% 284M 0s 2019-11-25T09:29:35.850 INFO:teuthology.orchestra.run.smithi153.stderr: 23750K .......... .......... .......... .......... .......... 32% 241M 0s 2019-11-25T09:29:35.851 INFO:teuthology.orchestra.run.smithi153.stderr: 23800K .......... .......... .......... .......... .......... 32% 278M 0s 2019-11-25T09:29:35.851 INFO:teuthology.orchestra.run.smithi153.stderr: 23850K .......... .......... .......... .......... .......... 32% 274M 0s 2019-11-25T09:29:35.851 INFO:teuthology.orchestra.run.smithi153.stderr: 23900K .......... .......... .......... .......... .......... 32% 299M 0s 2019-11-25T09:29:35.851 INFO:teuthology.orchestra.run.smithi153.stderr: 23950K .......... .......... .......... .......... .......... 32% 247M 0s 2019-11-25T09:29:35.851 INFO:teuthology.orchestra.run.smithi153.stderr: 24000K .......... .......... .......... .......... .......... 32% 269M 0s 2019-11-25T09:29:35.851 INFO:teuthology.orchestra.run.smithi153.stderr: 24050K .......... .......... .......... .......... .......... 32% 293M 0s 2019-11-25T09:29:35.851 INFO:teuthology.orchestra.run.smithi153.stderr: 24100K .......... .......... .......... .......... .......... 32% 298M 0s 2019-11-25T09:29:35.851 INFO:teuthology.orchestra.run.smithi153.stderr: 24150K .......... .......... .......... .......... .......... 32% 256M 0s 2019-11-25T09:29:35.852 INFO:teuthology.orchestra.run.smithi153.stderr: 24200K .......... .......... .......... .......... .......... 32% 302M 0s 2019-11-25T09:29:35.852 INFO:teuthology.orchestra.run.smithi153.stderr: 24250K .......... .......... .......... .......... .......... 33% 305M 0s 2019-11-25T09:29:35.852 INFO:teuthology.orchestra.run.smithi153.stderr: 24300K .......... .......... .......... .......... .......... 33% 307M 0s 2019-11-25T09:29:35.852 INFO:teuthology.orchestra.run.smithi153.stderr: 24350K .......... .......... .......... .......... .......... 33% 260M 0s 2019-11-25T09:29:35.852 INFO:teuthology.orchestra.run.smithi153.stderr: 24400K .......... .......... .......... .......... .......... 33% 289M 0s 2019-11-25T09:29:35.852 INFO:teuthology.orchestra.run.smithi153.stderr: 24450K .......... .......... .......... .......... .......... 33% 312M 0s 2019-11-25T09:29:35.852 INFO:teuthology.orchestra.run.smithi153.stderr: 24500K .......... .......... .......... .......... .......... 33% 305M 0s 2019-11-25T09:29:35.853 INFO:teuthology.orchestra.run.smithi153.stderr: 24550K .......... .......... .......... .......... .......... 33% 254M 0s 2019-11-25T09:29:35.853 INFO:teuthology.orchestra.run.smithi153.stderr: 24600K .......... .......... .......... .......... .......... 33% 298M 0s 2019-11-25T09:29:35.853 INFO:teuthology.orchestra.run.smithi153.stderr: 24650K .......... .......... .......... .......... .......... 33% 284M 0s 2019-11-25T09:29:35.853 INFO:teuthology.orchestra.run.smithi153.stderr: 24700K .......... .......... .......... .......... .......... 33% 286M 0s 2019-11-25T09:29:35.853 INFO:teuthology.orchestra.run.smithi153.stderr: 24750K .......... .......... .......... .......... .......... 33% 273M 0s 2019-11-25T09:29:35.853 INFO:teuthology.orchestra.run.smithi153.stderr: 24800K .......... .......... .......... .......... .......... 33% 285M 0s 2019-11-25T09:29:35.853 INFO:teuthology.orchestra.run.smithi153.stderr: 24850K .......... .......... .......... .......... .......... 33% 291M 0s 2019-11-25T09:29:35.854 INFO:teuthology.orchestra.run.smithi153.stderr: 24900K .......... .......... .......... .......... .......... 33% 275M 0s 2019-11-25T09:29:35.854 INFO:teuthology.orchestra.run.smithi153.stderr: 24950K .......... .......... .......... .......... .......... 33% 225M 0s 2019-11-25T09:29:35.854 INFO:teuthology.orchestra.run.smithi153.stderr: 25000K .......... .......... .......... .......... .......... 34% 275M 0s 2019-11-25T09:29:35.854 INFO:teuthology.orchestra.run.smithi153.stderr: 25050K .......... .......... .......... .......... .......... 34% 283M 0s 2019-11-25T09:29:35.854 INFO:teuthology.orchestra.run.smithi153.stderr: 25100K .......... .......... .......... .......... .......... 34% 275M 0s 2019-11-25T09:29:35.854 INFO:teuthology.orchestra.run.smithi153.stderr: 25150K .......... .......... .......... .......... .......... 34% 271M 0s 2019-11-25T09:29:35.854 INFO:teuthology.orchestra.run.smithi153.stderr: 25200K .......... .......... .......... .......... .......... 34% 264M 0s 2019-11-25T09:29:35.855 INFO:teuthology.orchestra.run.smithi153.stderr: 25250K .......... .......... .......... .......... .......... 34% 285M 0s 2019-11-25T09:29:35.855 INFO:teuthology.orchestra.run.smithi153.stderr: 25300K .......... .......... .......... .......... .......... 34% 319M 0s 2019-11-25T09:29:35.855 INFO:teuthology.orchestra.run.smithi153.stderr: 25350K .......... .......... .......... .......... .......... 34% 235M 0s 2019-11-25T09:29:35.855 INFO:teuthology.orchestra.run.smithi153.stderr: 25400K .......... .......... .......... .......... .......... 34% 289M 0s 2019-11-25T09:29:35.855 INFO:teuthology.orchestra.run.smithi153.stderr: 25450K .......... .......... .......... .......... .......... 34% 316M 0s 2019-11-25T09:29:35.855 INFO:teuthology.orchestra.run.smithi153.stderr: 25500K .......... .......... .......... .......... .......... 34% 320M 0s 2019-11-25T09:29:35.855 INFO:teuthology.orchestra.run.smithi153.stderr: 25550K .......... .......... .......... .......... .......... 34% 263M 0s 2019-11-25T09:29:35.856 INFO:teuthology.orchestra.run.smithi153.stderr: 25600K .......... .......... .......... .......... .......... 34% 319M 0s 2019-11-25T09:29:35.856 INFO:teuthology.orchestra.run.smithi153.stderr: 25650K .......... .......... .......... .......... .......... 34% 307M 0s 2019-11-25T09:29:35.856 INFO:teuthology.orchestra.run.smithi153.stderr: 25700K .......... .......... .......... .......... .......... 35% 303M 0s 2019-11-25T09:29:35.856 INFO:teuthology.orchestra.run.smithi153.stderr: 25750K .......... .......... .......... .......... .......... 35% 262M 0s 2019-11-25T09:29:35.856 INFO:teuthology.orchestra.run.smithi153.stderr: 25800K .......... .......... .......... .......... .......... 35% 302M 0s 2019-11-25T09:29:35.856 INFO:teuthology.orchestra.run.smithi153.stderr: 25850K .......... .......... .......... .......... .......... 35% 299M 0s 2019-11-25T09:29:35.856 INFO:teuthology.orchestra.run.smithi153.stderr: 25900K .......... .......... .......... .......... .......... 35% 301M 0s 2019-11-25T09:29:35.857 INFO:teuthology.orchestra.run.smithi153.stderr: 25950K .......... .......... .......... .......... .......... 35% 266M 0s 2019-11-25T09:29:35.857 INFO:teuthology.orchestra.run.smithi153.stderr: 26000K .......... .......... .......... .......... .......... 35% 309M 0s 2019-11-25T09:29:35.857 INFO:teuthology.orchestra.run.smithi153.stderr: 26050K .......... .......... .......... .......... .......... 35% 292M 0s 2019-11-25T09:29:35.857 INFO:teuthology.orchestra.run.smithi153.stderr: 26100K .......... .......... .......... .......... .......... 35% 296M 0s 2019-11-25T09:29:35.857 INFO:teuthology.orchestra.run.smithi153.stderr: 26150K .......... .......... .......... .......... .......... 35% 261M 0s 2019-11-25T09:29:35.857 INFO:teuthology.orchestra.run.smithi153.stderr: 26200K .......... .......... .......... .......... .......... 35% 300M 0s 2019-11-25T09:29:35.857 INFO:teuthology.orchestra.run.smithi153.stderr: 26250K .......... .......... .......... .......... .......... 35% 297M 0s 2019-11-25T09:29:35.858 INFO:teuthology.orchestra.run.smithi153.stderr: 26300K .......... .......... .......... .......... .......... 35% 303M 0s 2019-11-25T09:29:35.858 INFO:teuthology.orchestra.run.smithi153.stderr: 26350K .......... .......... .......... .......... .......... 35% 220M 0s 2019-11-25T09:29:35.858 INFO:teuthology.orchestra.run.smithi153.stderr: 26400K .......... .......... .......... .......... .......... 35% 273M 0s 2019-11-25T09:29:35.858 INFO:teuthology.orchestra.run.smithi153.stderr: 26450K .......... .......... .......... .......... .......... 36% 291M 0s 2019-11-25T09:29:35.858 INFO:teuthology.orchestra.run.smithi153.stderr: 26500K .......... .......... .......... .......... .......... 36% 260M 0s 2019-11-25T09:29:35.858 INFO:teuthology.orchestra.run.smithi153.stderr: 26550K .......... .......... .......... .......... .......... 36% 256M 0s 2019-11-25T09:29:35.858 INFO:teuthology.orchestra.run.smithi153.stderr: 26600K .......... .......... .......... .......... .......... 36% 267M 0s 2019-11-25T09:29:35.858 INFO:teuthology.orchestra.run.smithi153.stderr: 26650K .......... .......... .......... .......... .......... 36% 272M 0s 2019-11-25T09:29:35.859 INFO:teuthology.orchestra.run.smithi153.stderr: 26700K .......... .......... .......... .......... .......... 36% 310M 0s 2019-11-25T09:29:35.859 INFO:teuthology.orchestra.run.smithi153.stderr: 26750K .......... .......... .......... .......... .......... 36% 267M 0s 2019-11-25T09:29:35.859 INFO:teuthology.orchestra.run.smithi153.stderr: 26800K .......... .......... .......... .......... .......... 36% 301M 0s 2019-11-25T09:29:35.859 INFO:teuthology.orchestra.run.smithi153.stderr: 26850K .......... .......... .......... .......... .......... 36% 318M 0s 2019-11-25T09:29:35.859 INFO:teuthology.orchestra.run.smithi153.stderr: 26900K .......... .......... .......... .......... .......... 36% 306M 0s 2019-11-25T09:29:35.859 INFO:teuthology.orchestra.run.smithi153.stderr: 26950K .......... .......... .......... .......... .......... 36% 248M 0s 2019-11-25T09:29:35.859 INFO:teuthology.orchestra.run.smithi153.stderr: 27000K .......... .......... .......... .......... .......... 36% 326M 0s 2019-11-25T09:29:35.860 INFO:teuthology.orchestra.run.smithi153.stderr: 27050K .......... .......... .......... .......... .......... 36% 310M 0s 2019-11-25T09:29:35.860 INFO:teuthology.orchestra.run.smithi153.stderr: 27100K .......... .......... .......... .......... .......... 36% 295M 0s 2019-11-25T09:29:35.860 INFO:teuthology.orchestra.run.smithi153.stderr: 27150K .......... .......... .......... .......... .......... 36% 269M 0s 2019-11-25T09:29:35.860 INFO:teuthology.orchestra.run.smithi153.stderr: 27200K .......... .......... .......... .......... .......... 37% 315M 0s 2019-11-25T09:29:35.860 INFO:teuthology.orchestra.run.smithi153.stderr: 27250K .......... .......... .......... .......... .......... 37% 293M 0s 2019-11-25T09:29:35.860 INFO:teuthology.orchestra.run.smithi153.stderr: 27300K .......... .......... .......... .......... .......... 37% 289M 0s 2019-11-25T09:29:35.860 INFO:teuthology.orchestra.run.smithi153.stderr: 27350K .......... .......... .......... .......... .......... 37% 239M 0s 2019-11-25T09:29:35.861 INFO:teuthology.orchestra.run.smithi153.stderr: 27400K .......... .......... .......... .......... .......... 37% 315M 0s 2019-11-25T09:29:35.861 INFO:teuthology.orchestra.run.smithi153.stderr: 27450K .......... .......... .......... .......... .......... 37% 290M 0s 2019-11-25T09:29:35.861 INFO:teuthology.orchestra.run.smithi153.stderr: 27500K .......... .......... .......... .......... .......... 37% 310M 0s 2019-11-25T09:29:35.861 INFO:teuthology.orchestra.run.smithi153.stderr: 27550K .......... .......... .......... .......... .......... 37% 262M 0s 2019-11-25T09:29:35.861 INFO:teuthology.orchestra.run.smithi153.stderr: 27600K .......... .......... .......... .......... .......... 37% 310M 0s 2019-11-25T09:29:35.861 INFO:teuthology.orchestra.run.smithi153.stderr: 27650K .......... .......... .......... .......... .......... 37% 299M 0s 2019-11-25T09:29:35.861 INFO:teuthology.orchestra.run.smithi153.stderr: 27700K .......... .......... .......... .......... .......... 37% 340M 0s 2019-11-25T09:29:35.862 INFO:teuthology.orchestra.run.smithi153.stderr: 27750K .......... .......... .......... .......... .......... 37% 233M 0s 2019-11-25T09:29:35.862 INFO:teuthology.orchestra.run.smithi153.stderr: 27800K .......... .......... .......... .......... .......... 37% 303M 0s 2019-11-25T09:29:35.862 INFO:teuthology.orchestra.run.smithi153.stderr: 27850K .......... .......... .......... .......... .......... 37% 301M 0s 2019-11-25T09:29:35.862 INFO:teuthology.orchestra.run.smithi153.stderr: 27900K .......... .......... .......... .......... .......... 37% 289M 0s 2019-11-25T09:29:35.862 INFO:teuthology.orchestra.run.smithi153.stderr: 27950K .......... .......... .......... .......... .......... 38% 264M 0s 2019-11-25T09:29:35.862 INFO:teuthology.orchestra.run.smithi153.stderr: 28000K .......... .......... .......... .......... .......... 38% 277M 0s 2019-11-25T09:29:35.862 INFO:teuthology.orchestra.run.smithi153.stderr: 28050K .......... .......... .......... .......... .......... 38% 310M 0s 2019-11-25T09:29:35.863 INFO:teuthology.orchestra.run.smithi153.stderr: 28100K .......... .......... .......... .......... .......... 38% 290M 0s 2019-11-25T09:29:35.863 INFO:teuthology.orchestra.run.smithi153.stderr: 28150K .......... .......... .......... .......... .......... 38% 244M 0s 2019-11-25T09:29:35.863 INFO:teuthology.orchestra.run.smithi153.stderr: 28200K .......... .......... .......... .......... .......... 38% 289M 0s 2019-11-25T09:29:35.863 INFO:teuthology.orchestra.run.smithi153.stderr: 28250K .......... .......... .......... .......... .......... 38% 287M 0s 2019-11-25T09:29:35.863 INFO:teuthology.orchestra.run.smithi153.stderr: 28300K .......... .......... .......... .......... .......... 38% 325M 0s 2019-11-25T09:29:35.863 INFO:teuthology.orchestra.run.smithi153.stderr: 28350K .......... .......... .......... .......... .......... 38% 285M 0s 2019-11-25T09:29:35.863 INFO:teuthology.orchestra.run.smithi153.stderr: 28400K .......... .......... .......... .......... .......... 38% 317M 0s 2019-11-25T09:29:35.864 INFO:teuthology.orchestra.run.smithi153.stderr: 28450K .......... .......... .......... .......... .......... 38% 299M 0s 2019-11-25T09:29:35.864 INFO:teuthology.orchestra.run.smithi153.stderr: 28500K .......... .......... .......... .......... .......... 38% 222M 0s 2019-11-25T09:29:35.864 INFO:teuthology.orchestra.run.smithi153.stderr: 28550K .......... .......... .......... .......... .......... 38% 229M 0s 2019-11-25T09:29:35.864 INFO:teuthology.orchestra.run.smithi153.stderr: 28600K .......... .......... .......... .......... .......... 38% 280M 0s 2019-11-25T09:29:35.864 INFO:teuthology.orchestra.run.smithi153.stderr: 28650K .......... .......... .......... .......... .......... 39% 301M 0s 2019-11-25T09:29:35.864 INFO:teuthology.orchestra.run.smithi153.stderr: 28700K .......... .......... .......... .......... .......... 39% 273M 0s 2019-11-25T09:29:35.864 INFO:teuthology.orchestra.run.smithi153.stderr: 28750K .......... .......... .......... .......... .......... 39% 259M 0s 2019-11-25T09:29:35.865 INFO:teuthology.orchestra.run.smithi153.stderr: 28800K .......... .......... .......... .......... .......... 39% 311M 0s 2019-11-25T09:29:35.865 INFO:teuthology.orchestra.run.smithi153.stderr: 28850K .......... .......... .......... .......... .......... 39% 298M 0s 2019-11-25T09:29:35.865 INFO:teuthology.orchestra.run.smithi153.stderr: 28900K .......... .......... .......... .......... .......... 39% 316M 0s 2019-11-25T09:29:35.865 INFO:teuthology.orchestra.run.smithi153.stderr: 28950K .......... .......... .......... .......... .......... 39% 245M 0s 2019-11-25T09:29:35.865 INFO:teuthology.orchestra.run.smithi153.stderr: 29000K .......... .......... .......... .......... .......... 39% 256M 0s 2019-11-25T09:29:35.865 INFO:teuthology.orchestra.run.smithi153.stderr: 29050K .......... .......... .......... .......... .......... 39% 276M 0s 2019-11-25T09:29:35.865 INFO:teuthology.orchestra.run.smithi153.stderr: 29100K .......... .......... .......... .......... .......... 39% 307M 0s 2019-11-25T09:29:35.866 INFO:teuthology.orchestra.run.smithi153.stderr: 29150K .......... .......... .......... .......... .......... 39% 287M 0s 2019-11-25T09:29:35.866 INFO:teuthology.orchestra.run.smithi153.stderr: 29200K .......... .......... .......... .......... .......... 39% 324M 0s 2019-11-25T09:29:35.866 INFO:teuthology.orchestra.run.smithi153.stderr: 29250K .......... .......... .......... .......... .......... 39% 312M 0s 2019-11-25T09:29:35.866 INFO:teuthology.orchestra.run.smithi153.stderr: 29300K .......... .......... .......... .......... .......... 39% 279M 0s 2019-11-25T09:29:35.866 INFO:teuthology.orchestra.run.smithi153.stderr: 29350K .......... .......... .......... .......... .......... 39% 235M 0s 2019-11-25T09:29:35.866 INFO:teuthology.orchestra.run.smithi153.stderr: 29400K .......... .......... .......... .......... .......... 40% 276M 0s 2019-11-25T09:29:35.866 INFO:teuthology.orchestra.run.smithi153.stderr: 29450K .......... .......... .......... .......... .......... 40% 303M 0s 2019-11-25T09:29:35.867 INFO:teuthology.orchestra.run.smithi153.stderr: 29500K .......... .......... .......... .......... .......... 40% 291M 0s 2019-11-25T09:29:35.867 INFO:teuthology.orchestra.run.smithi153.stderr: 29550K .......... .......... .......... .......... .......... 40% 285M 0s 2019-11-25T09:29:35.867 INFO:teuthology.orchestra.run.smithi153.stderr: 29600K .......... .......... .......... .......... .......... 40% 312M 0s 2019-11-25T09:29:35.867 INFO:teuthology.orchestra.run.smithi153.stderr: 29650K .......... .......... .......... .......... .......... 40% 302M 0s 2019-11-25T09:29:35.867 INFO:teuthology.orchestra.run.smithi153.stderr: 29700K .......... .......... .......... .......... .......... 40% 300M 0s 2019-11-25T09:29:35.867 INFO:teuthology.orchestra.run.smithi153.stderr: 29750K .......... .......... .......... .......... .......... 40% 250M 0s 2019-11-25T09:29:35.867 INFO:teuthology.orchestra.run.smithi153.stderr: 29800K .......... .......... .......... .......... .......... 40% 302M 0s 2019-11-25T09:29:35.867 INFO:teuthology.orchestra.run.smithi153.stderr: 29850K .......... .......... .......... .......... .......... 40% 299M 0s 2019-11-25T09:29:35.868 INFO:teuthology.orchestra.run.smithi153.stderr: 29900K .......... .......... .......... .......... .......... 40% 311M 0s 2019-11-25T09:29:35.868 INFO:teuthology.orchestra.run.smithi153.stderr: 29950K .......... .......... .......... .......... .......... 40% 258M 0s 2019-11-25T09:29:35.868 INFO:teuthology.orchestra.run.smithi153.stderr: 30000K .......... .......... .......... .......... .......... 40% 320M 0s 2019-11-25T09:29:35.868 INFO:teuthology.orchestra.run.smithi153.stderr: 30050K .......... .......... .......... .......... .......... 40% 301M 0s 2019-11-25T09:29:35.868 INFO:teuthology.orchestra.run.smithi153.stderr: 30100K .......... .......... .......... .......... .......... 40% 281M 0s 2019-11-25T09:29:35.868 INFO:teuthology.orchestra.run.smithi153.stderr: 30150K .......... .......... .......... .......... .......... 41% 228M 0s 2019-11-25T09:29:35.868 INFO:teuthology.orchestra.run.smithi153.stderr: 30200K .......... .......... .......... .......... .......... 41% 319M 0s 2019-11-25T09:29:35.869 INFO:teuthology.orchestra.run.smithi153.stderr: 30250K .......... .......... .......... .......... .......... 41% 299M 0s 2019-11-25T09:29:35.869 INFO:teuthology.orchestra.run.smithi153.stderr: 30300K .......... .......... .......... .......... .......... 41% 283M 0s 2019-11-25T09:29:35.869 INFO:teuthology.orchestra.run.smithi153.stderr: 30350K .......... .......... .......... .......... .......... 41% 241M 0s 2019-11-25T09:29:35.869 INFO:teuthology.orchestra.run.smithi153.stderr: 30400K .......... .......... .......... .......... .......... 41% 291M 0s 2019-11-25T09:29:35.869 INFO:teuthology.orchestra.run.smithi153.stderr: 30450K .......... .......... .......... .......... .......... 41% 303M 0s 2019-11-25T09:29:35.869 INFO:teuthology.orchestra.run.smithi153.stderr: 30500K .......... .......... .......... .......... .......... 41% 291M 0s 2019-11-25T09:29:35.869 INFO:teuthology.orchestra.run.smithi153.stderr: 30550K .......... .......... .......... .......... .......... 41% 254M 0s 2019-11-25T09:29:35.870 INFO:teuthology.orchestra.run.smithi153.stderr: 30600K .......... .......... .......... .......... .......... 41% 330M 0s 2019-11-25T09:29:35.870 INFO:teuthology.orchestra.run.smithi153.stderr: 30650K .......... .......... .......... .......... .......... 41% 315M 0s 2019-11-25T09:29:35.870 INFO:teuthology.orchestra.run.smithi153.stderr: 30700K .......... .......... .......... .......... .......... 41% 305M 0s 2019-11-25T09:29:35.870 INFO:teuthology.orchestra.run.smithi153.stderr: 30750K .......... .......... .......... .......... .......... 41% 262M 0s 2019-11-25T09:29:35.870 INFO:teuthology.orchestra.run.smithi153.stderr: 30800K .......... .......... .......... .......... .......... 41% 312M 0s 2019-11-25T09:29:35.870 INFO:teuthology.orchestra.run.smithi153.stderr: 30850K .......... .......... .......... .......... .......... 42% 331M 0s 2019-11-25T09:29:35.870 INFO:teuthology.orchestra.run.smithi153.stderr: 30900K .......... .......... .......... .......... .......... 42% 327M 0s 2019-11-25T09:29:35.871 INFO:teuthology.orchestra.run.smithi153.stderr: 30950K .......... .......... .......... .......... .......... 42% 283M 0s 2019-11-25T09:29:35.871 INFO:teuthology.orchestra.run.smithi153.stderr: 31000K .......... .......... .......... .......... .......... 42% 282M 0s 2019-11-25T09:29:35.871 INFO:teuthology.orchestra.run.smithi153.stderr: 31050K .......... .......... .......... .......... .......... 42% 310M 0s 2019-11-25T09:29:35.871 INFO:teuthology.orchestra.run.smithi153.stderr: 31100K .......... .......... .......... .......... .......... 42% 341M 0s 2019-11-25T09:29:35.871 INFO:teuthology.orchestra.run.smithi153.stderr: 31150K .......... .......... .......... .......... .......... 42% 268M 0s 2019-11-25T09:29:35.871 INFO:teuthology.orchestra.run.smithi153.stderr: 31200K .......... .......... .......... .......... .......... 42% 300M 0s 2019-11-25T09:29:35.871 INFO:teuthology.orchestra.run.smithi153.stderr: 31250K .......... .......... .......... .......... .......... 42% 287M 0s 2019-11-25T09:29:35.872 INFO:teuthology.orchestra.run.smithi153.stderr: 31300K .......... .......... .......... .......... .......... 42% 298M 0s 2019-11-25T09:29:35.872 INFO:teuthology.orchestra.run.smithi153.stderr: 31350K .......... .......... .......... .......... .......... 42% 240M 0s 2019-11-25T09:29:35.872 INFO:teuthology.orchestra.run.smithi153.stderr: 31400K .......... .......... .......... .......... .......... 42% 297M 0s 2019-11-25T09:29:35.872 INFO:teuthology.orchestra.run.smithi153.stderr: 31450K .......... .......... .......... .......... .......... 42% 321M 0s 2019-11-25T09:29:35.872 INFO:teuthology.orchestra.run.smithi153.stderr: 31500K .......... .......... .......... .......... .......... 42% 297M 0s 2019-11-25T09:29:35.872 INFO:teuthology.orchestra.run.smithi153.stderr: 31550K .......... .......... .......... .......... .......... 42% 280M 0s 2019-11-25T09:29:35.872 INFO:teuthology.orchestra.run.smithi153.stderr: 31600K .......... .......... .......... .......... .......... 43% 333M 0s 2019-11-25T09:29:35.873 INFO:teuthology.orchestra.run.smithi153.stderr: 31650K .......... .......... .......... .......... .......... 43% 288M 0s 2019-11-25T09:29:35.873 INFO:teuthology.orchestra.run.smithi153.stderr: 31700K .......... .......... .......... .......... .......... 43% 346M 0s 2019-11-25T09:29:35.873 INFO:teuthology.orchestra.run.smithi153.stderr: 31750K .......... .......... .......... .......... .......... 43% 283M 0s 2019-11-25T09:29:35.873 INFO:teuthology.orchestra.run.smithi153.stderr: 31800K .......... .......... .......... .......... .......... 43% 331M 0s 2019-11-25T09:29:35.873 INFO:teuthology.orchestra.run.smithi153.stderr: 31850K .......... .......... .......... .......... .......... 43% 335M 0s 2019-11-25T09:29:35.873 INFO:teuthology.orchestra.run.smithi153.stderr: 31900K .......... .......... .......... .......... .......... 43% 338M 0s 2019-11-25T09:29:35.873 INFO:teuthology.orchestra.run.smithi153.stderr: 31950K .......... .......... .......... .......... .......... 43% 279M 0s 2019-11-25T09:29:35.873 INFO:teuthology.orchestra.run.smithi153.stderr: 32000K .......... .......... .......... .......... .......... 43% 288M 0s 2019-11-25T09:29:35.874 INFO:teuthology.orchestra.run.smithi153.stderr: 32050K .......... .......... .......... .......... .......... 43% 324M 0s 2019-11-25T09:29:35.874 INFO:teuthology.orchestra.run.smithi153.stderr: 32100K .......... .......... .......... .......... .......... 43% 327M 0s 2019-11-25T09:29:35.874 INFO:teuthology.orchestra.run.smithi153.stderr: 32150K .......... .......... .......... .......... .......... 43% 289M 0s 2019-11-25T09:29:35.874 INFO:teuthology.orchestra.run.smithi153.stderr: 32200K .......... .......... .......... .......... .......... 43% 336M 0s 2019-11-25T09:29:35.874 INFO:teuthology.orchestra.run.smithi153.stderr: 32250K .......... .......... .......... .......... .......... 43% 323M 0s 2019-11-25T09:29:35.874 INFO:teuthology.orchestra.run.smithi153.stderr: 32300K .......... .......... .......... .......... .......... 43% 337M 0s 2019-11-25T09:29:35.874 INFO:teuthology.orchestra.run.smithi153.stderr: 32350K .......... .......... .......... .......... .......... 44% 294M 0s 2019-11-25T09:29:35.875 INFO:teuthology.orchestra.run.smithi153.stderr: 32400K .......... .......... .......... .......... .......... 44% 332M 0s 2019-11-25T09:29:35.875 INFO:teuthology.orchestra.run.smithi153.stderr: 32450K .......... .......... .......... .......... .......... 44% 334M 0s 2019-11-25T09:29:35.875 INFO:teuthology.orchestra.run.smithi153.stderr: 32500K .......... .......... .......... .......... .......... 44% 339M 0s 2019-11-25T09:29:35.875 INFO:teuthology.orchestra.run.smithi153.stderr: 32550K .......... .......... .......... .......... .......... 44% 277M 0s 2019-11-25T09:29:35.875 INFO:teuthology.orchestra.run.smithi153.stderr: 32600K .......... .......... .......... .......... .......... 44% 326M 0s 2019-11-25T09:29:35.875 INFO:teuthology.orchestra.run.smithi153.stderr: 32650K .......... .......... .......... .......... .......... 44% 331M 0s 2019-11-25T09:29:35.875 INFO:teuthology.orchestra.run.smithi153.stderr: 32700K .......... .......... .......... .......... .......... 44% 333M 0s 2019-11-25T09:29:35.876 INFO:teuthology.orchestra.run.smithi153.stderr: 32750K .......... .......... .......... .......... .......... 44% 294M 0s 2019-11-25T09:29:35.876 INFO:teuthology.orchestra.run.smithi153.stderr: 32800K .......... .......... .......... .......... .......... 44% 336M 0s 2019-11-25T09:29:35.876 INFO:teuthology.orchestra.run.smithi153.stderr: 32850K .......... .......... .......... .......... .......... 44% 332M 0s 2019-11-25T09:29:35.876 INFO:teuthology.orchestra.run.smithi153.stderr: 32900K .......... .......... .......... .......... .......... 44% 333M 0s 2019-11-25T09:29:35.876 INFO:teuthology.orchestra.run.smithi153.stderr: 32950K .......... .......... .......... .......... .......... 44% 254M 0s 2019-11-25T09:29:35.876 INFO:teuthology.orchestra.run.smithi153.stderr: 33000K .......... .......... .......... .......... .......... 44% 321M 0s 2019-11-25T09:29:35.876 INFO:teuthology.orchestra.run.smithi153.stderr: 33050K .......... .......... .......... .......... .......... 44% 289M 0s 2019-11-25T09:29:35.877 INFO:teuthology.orchestra.run.smithi153.stderr: 33100K .......... .......... .......... .......... .......... 45% 325M 0s 2019-11-25T09:29:35.877 INFO:teuthology.orchestra.run.smithi153.stderr: 33150K .......... .......... .......... .......... .......... 45% 274M 0s 2019-11-25T09:29:35.877 INFO:teuthology.orchestra.run.smithi153.stderr: 33200K .......... .......... .......... .......... .......... 45% 297M 0s 2019-11-25T09:29:35.877 INFO:teuthology.orchestra.run.smithi153.stderr: 33250K .......... .......... .......... .......... .......... 45% 292M 0s 2019-11-25T09:29:35.877 INFO:teuthology.orchestra.run.smithi153.stderr: 33300K .......... .......... .......... .......... .......... 45% 335M 0s 2019-11-25T09:29:35.877 INFO:teuthology.orchestra.run.smithi153.stderr: 33350K .......... .......... .......... .......... .......... 45% 263M 0s 2019-11-25T09:29:35.877 INFO:teuthology.orchestra.run.smithi153.stderr: 33400K .......... .......... .......... .......... .......... 45% 347M 0s 2019-11-25T09:29:35.878 INFO:teuthology.orchestra.run.smithi153.stderr: 33450K .......... .......... .......... .......... .......... 45% 311M 0s 2019-11-25T09:29:35.878 INFO:teuthology.orchestra.run.smithi153.stderr: 33500K .......... .......... .......... .......... .......... 45% 315M 0s 2019-11-25T09:29:35.878 INFO:teuthology.orchestra.run.smithi153.stderr: 33550K .......... .......... .......... .......... .......... 45% 281M 0s 2019-11-25T09:29:35.878 INFO:teuthology.orchestra.run.smithi153.stderr: 33600K .......... .......... .......... .......... .......... 45% 305M 0s 2019-11-25T09:29:35.878 INFO:teuthology.orchestra.run.smithi153.stderr: 33650K .......... .......... .......... .......... .......... 45% 309M 0s 2019-11-25T09:29:35.878 INFO:teuthology.orchestra.run.smithi153.stderr: 33700K .......... .......... .......... .......... .......... 45% 306M 0s 2019-11-25T09:29:35.878 INFO:teuthology.orchestra.run.smithi153.stderr: 33750K .......... .......... .......... .......... .......... 45% 265M 0s 2019-11-25T09:29:35.879 INFO:teuthology.orchestra.run.smithi153.stderr: 33800K .......... .......... .......... .......... .......... 46% 318M 0s 2019-11-25T09:29:35.879 INFO:teuthology.orchestra.run.smithi153.stderr: 33850K .......... .......... .......... .......... .......... 46% 316M 0s 2019-11-25T09:29:35.879 INFO:teuthology.orchestra.run.smithi153.stderr: 33900K .......... .......... .......... .......... .......... 46% 317M 0s 2019-11-25T09:29:35.879 INFO:teuthology.orchestra.run.smithi153.stderr: 33950K .......... .......... .......... .......... .......... 46% 292M 0s 2019-11-25T09:29:35.879 INFO:teuthology.orchestra.run.smithi153.stderr: 34000K .......... .......... .......... .......... .......... 46% 336M 0s 2019-11-25T09:29:35.879 INFO:teuthology.orchestra.run.smithi153.stderr: 34050K .......... .......... .......... .......... .......... 46% 331M 0s 2019-11-25T09:29:35.879 INFO:teuthology.orchestra.run.smithi153.stderr: 34100K .......... .......... .......... .......... .......... 46% 338M 0s 2019-11-25T09:29:35.880 INFO:teuthology.orchestra.run.smithi153.stderr: 34150K .......... .......... .......... .......... .......... 46% 275M 0s 2019-11-25T09:29:35.880 INFO:teuthology.orchestra.run.smithi153.stderr: 34200K .......... .......... .......... .......... .......... 46% 317M 0s 2019-11-25T09:29:35.880 INFO:teuthology.orchestra.run.smithi153.stderr: 34250K .......... .......... .......... .......... .......... 46% 328M 0s 2019-11-25T09:29:35.880 INFO:teuthology.orchestra.run.smithi153.stderr: 34300K .......... .......... .......... .......... .......... 46% 321M 0s 2019-11-25T09:29:35.880 INFO:teuthology.orchestra.run.smithi153.stderr: 34350K .......... .......... .......... .......... .......... 46% 282M 0s 2019-11-25T09:29:35.880 INFO:teuthology.orchestra.run.smithi153.stderr: 34400K .......... .......... .......... .......... .......... 46% 346M 0s 2019-11-25T09:29:35.880 INFO:teuthology.orchestra.run.smithi153.stderr: 34450K .......... .......... .......... .......... .......... 46% 337M 0s 2019-11-25T09:29:35.881 INFO:teuthology.orchestra.run.smithi153.stderr: 34500K .......... .......... .......... .......... .......... 46% 331M 0s 2019-11-25T09:29:35.881 INFO:teuthology.orchestra.run.smithi153.stderr: 34550K .......... .......... .......... .......... .......... 47% 279M 0s 2019-11-25T09:29:35.881 INFO:teuthology.orchestra.run.smithi153.stderr: 34600K .......... .......... .......... .......... .......... 47% 343M 0s 2019-11-25T09:29:35.881 INFO:teuthology.orchestra.run.smithi153.stderr: 34650K .......... .......... .......... .......... .......... 47% 325M 0s 2019-11-25T09:29:35.881 INFO:teuthology.orchestra.run.smithi153.stderr: 34700K .......... .......... .......... .......... .......... 47% 331M 0s 2019-11-25T09:29:35.881 INFO:teuthology.orchestra.run.smithi153.stderr: 34750K .......... .......... .......... .......... .......... 47% 311M 0s 2019-11-25T09:29:35.881 INFO:teuthology.orchestra.run.smithi153.stderr: 34800K .......... .......... .......... .......... .......... 47% 339M 0s 2019-11-25T09:29:35.881 INFO:teuthology.orchestra.run.smithi153.stderr: 34850K .......... .......... .......... .......... .......... 47% 330M 0s 2019-11-25T09:29:35.882 INFO:teuthology.orchestra.run.smithi153.stderr: 34900K .......... .......... .......... .......... .......... 47% 298M 0s 2019-11-25T09:29:35.882 INFO:teuthology.orchestra.run.smithi153.stderr: 34950K .......... .......... .......... .......... .......... 47% 266M 0s 2019-11-25T09:29:35.882 INFO:teuthology.orchestra.run.smithi153.stderr: 35000K .......... .......... .......... .......... .......... 47% 329M 0s 2019-11-25T09:29:35.882 INFO:teuthology.orchestra.run.smithi153.stderr: 35050K .......... .......... .......... .......... .......... 47% 330M 0s 2019-11-25T09:29:35.882 INFO:teuthology.orchestra.run.smithi153.stderr: 35100K .......... .......... .......... .......... .......... 47% 340M 0s 2019-11-25T09:29:35.882 INFO:teuthology.orchestra.run.smithi153.stderr: 35150K .......... .......... .......... .......... .......... 47% 284M 0s 2019-11-25T09:29:35.882 INFO:teuthology.orchestra.run.smithi153.stderr: 35200K .......... .......... .......... .......... .......... 47% 311M 0s 2019-11-25T09:29:35.883 INFO:teuthology.orchestra.run.smithi153.stderr: 35250K .......... .......... .......... .......... .......... 47% 289M 0s 2019-11-25T09:29:35.883 INFO:teuthology.orchestra.run.smithi153.stderr: 35300K .......... .......... .......... .......... .......... 48% 344M 0s 2019-11-25T09:29:35.883 INFO:teuthology.orchestra.run.smithi153.stderr: 35350K .......... .......... .......... .......... .......... 48% 250M 0s 2019-11-25T09:29:35.883 INFO:teuthology.orchestra.run.smithi153.stderr: 35400K .......... .......... .......... .......... .......... 48% 317M 0s 2019-11-25T09:29:35.883 INFO:teuthology.orchestra.run.smithi153.stderr: 35450K .......... .......... .......... .......... .......... 48% 320M 0s 2019-11-25T09:29:35.883 INFO:teuthology.orchestra.run.smithi153.stderr: 35500K .......... .......... .......... .......... .......... 48% 315M 0s 2019-11-25T09:29:35.883 INFO:teuthology.orchestra.run.smithi153.stderr: 35550K .......... .......... .......... .......... .......... 48% 269M 0s 2019-11-25T09:29:35.884 INFO:teuthology.orchestra.run.smithi153.stderr: 35600K .......... .......... .......... .......... .......... 48% 316M 0s 2019-11-25T09:29:35.884 INFO:teuthology.orchestra.run.smithi153.stderr: 35650K .......... .......... .......... .......... .......... 48% 329M 0s 2019-11-25T09:29:35.884 INFO:teuthology.orchestra.run.smithi153.stderr: 35700K .......... .......... .......... .......... .......... 48% 347M 0s 2019-11-25T09:29:35.884 INFO:teuthology.orchestra.run.smithi153.stderr: 35750K .......... .......... .......... .......... .......... 48% 281M 0s 2019-11-25T09:29:35.884 INFO:teuthology.orchestra.run.smithi153.stderr: 35800K .......... .......... .......... .......... .......... 48% 328M 0s 2019-11-25T09:29:35.884 INFO:teuthology.orchestra.run.smithi153.stderr: 35850K .......... .......... .......... .......... .......... 48% 351M 0s 2019-11-25T09:29:35.884 INFO:teuthology.orchestra.run.smithi153.stderr: 35900K .......... .......... .......... .......... .......... 48% 345M 0s 2019-11-25T09:29:35.885 INFO:teuthology.orchestra.run.smithi153.stderr: 35950K .......... .......... .......... .......... .......... 48% 289M 0s 2019-11-25T09:29:35.885 INFO:teuthology.orchestra.run.smithi153.stderr: 36000K .......... .......... .......... .......... .......... 49% 329M 0s 2019-11-25T09:29:35.885 INFO:teuthology.orchestra.run.smithi153.stderr: 36050K .......... .......... .......... .......... .......... 49% 329M 0s 2019-11-25T09:29:35.885 INFO:teuthology.orchestra.run.smithi153.stderr: 36100K .......... .......... .......... .......... .......... 49% 334M 0s 2019-11-25T09:29:35.885 INFO:teuthology.orchestra.run.smithi153.stderr: 36150K .......... .......... .......... .......... .......... 49% 279M 0s 2019-11-25T09:29:35.885 INFO:teuthology.orchestra.run.smithi153.stderr: 36200K .......... .......... .......... .......... .......... 49% 333M 0s 2019-11-25T09:29:35.885 INFO:teuthology.orchestra.run.smithi153.stderr: 36250K .......... .......... .......... .......... .......... 49% 344M 0s 2019-11-25T09:29:35.886 INFO:teuthology.orchestra.run.smithi153.stderr: 36300K .......... .......... .......... .......... .......... 49% 343M 0s 2019-11-25T09:29:35.886 INFO:teuthology.orchestra.run.smithi153.stderr: 36350K .......... .......... .......... .......... .......... 49% 294M 0s 2019-11-25T09:29:35.886 INFO:teuthology.orchestra.run.smithi153.stderr: 36400K .......... .......... .......... .......... .......... 49% 333M 0s 2019-11-25T09:29:35.886 INFO:teuthology.orchestra.run.smithi153.stderr: 36450K .......... .......... .......... .......... .......... 49% 331M 0s 2019-11-25T09:29:35.886 INFO:teuthology.orchestra.run.smithi153.stderr: 36500K .......... .......... .......... .......... .......... 49% 336M 0s 2019-11-25T09:29:35.886 INFO:teuthology.orchestra.run.smithi153.stderr: 36550K .......... .......... .......... .......... .......... 49% 282M 0s 2019-11-25T09:29:35.886 INFO:teuthology.orchestra.run.smithi153.stderr: 36600K .......... .......... .......... .......... .......... 49% 305M 0s 2019-11-25T09:29:35.887 INFO:teuthology.orchestra.run.smithi153.stderr: 36650K .......... .......... .......... .......... .......... 49% 306M 0s 2019-11-25T09:29:35.887 INFO:teuthology.orchestra.run.smithi153.stderr: 36700K .......... .......... .......... .......... .......... 49% 316M 0s 2019-11-25T09:29:35.887 INFO:teuthology.orchestra.run.smithi153.stderr: 36750K .......... .......... .......... .......... .......... 50% 289M 0s 2019-11-25T09:29:35.887 INFO:teuthology.orchestra.run.smithi153.stderr: 36800K .......... .......... .......... .......... .......... 50% 315M 0s 2019-11-25T09:29:35.887 INFO:teuthology.orchestra.run.smithi153.stderr: 36850K .......... .......... .......... .......... .......... 50% 322M 0s 2019-11-25T09:29:35.887 INFO:teuthology.orchestra.run.smithi153.stderr: 36900K .......... .......... .......... .......... .......... 50% 327M 0s 2019-11-25T09:29:35.887 INFO:teuthology.orchestra.run.smithi153.stderr: 36950K .......... .......... .......... .......... .......... 50% 269M 0s 2019-11-25T09:29:35.887 INFO:teuthology.orchestra.run.smithi153.stderr: 37000K .......... .......... .......... .......... .......... 50% 347M 0s 2019-11-25T09:29:35.888 INFO:teuthology.orchestra.run.smithi153.stderr: 37050K .......... .......... .......... .......... .......... 50% 342M 0s 2019-11-25T09:29:35.888 INFO:teuthology.orchestra.run.smithi153.stderr: 37100K .......... .......... .......... .......... .......... 50% 303M 0s 2019-11-25T09:29:35.888 INFO:teuthology.orchestra.run.smithi153.stderr: 37150K .......... .......... .......... .......... .......... 50% 268M 0s 2019-11-25T09:29:35.888 INFO:teuthology.orchestra.run.smithi153.stderr: 37200K .......... .......... .......... .......... .......... 50% 330M 0s 2019-11-25T09:29:35.888 INFO:teuthology.orchestra.run.smithi153.stderr: 37250K .......... .......... .......... .......... .......... 50% 288M 0s 2019-11-25T09:29:35.888 INFO:teuthology.orchestra.run.smithi153.stderr: 37300K .......... .......... .......... .......... .......... 50% 300M 0s 2019-11-25T09:29:35.888 INFO:teuthology.orchestra.run.smithi153.stderr: 37350K .......... .......... .......... .......... .......... 50% 254M 0s 2019-11-25T09:29:35.889 INFO:teuthology.orchestra.run.smithi153.stderr: 37400K .......... .......... .......... .......... .......... 50% 304M 0s 2019-11-25T09:29:35.889 INFO:teuthology.orchestra.run.smithi153.stderr: 37450K .......... .......... .......... .......... .......... 50% 317M 0s 2019-11-25T09:29:35.889 INFO:teuthology.orchestra.run.smithi153.stderr: 37500K .......... .......... .......... .......... .......... 51% 336M 0s 2019-11-25T09:29:35.889 INFO:teuthology.orchestra.run.smithi153.stderr: 37550K .......... .......... .......... .......... .......... 51% 302M 0s 2019-11-25T09:29:35.889 INFO:teuthology.orchestra.run.smithi153.stderr: 37600K .......... .......... .......... .......... .......... 51% 326M 0s 2019-11-25T09:29:35.889 INFO:teuthology.orchestra.run.smithi153.stderr: 37650K .......... .......... .......... .......... .......... 51% 345M 0s 2019-11-25T09:29:35.889 INFO:teuthology.orchestra.run.smithi153.stderr: 37700K .......... .......... .......... .......... .......... 51% 346M 0s 2019-11-25T09:29:35.890 INFO:teuthology.orchestra.run.smithi153.stderr: 37750K .......... .......... .......... .......... .......... 51% 297M 0s 2019-11-25T09:29:35.890 INFO:teuthology.orchestra.run.smithi153.stderr: 37800K .......... .......... .......... .......... .......... 51% 349M 0s 2019-11-25T09:29:35.890 INFO:teuthology.orchestra.run.smithi153.stderr: 37850K .......... .......... .......... .......... .......... 51% 332M 0s 2019-11-25T09:29:35.890 INFO:teuthology.orchestra.run.smithi153.stderr: 37900K .......... .......... .......... .......... .......... 51% 365M 0s 2019-11-25T09:29:35.890 INFO:teuthology.orchestra.run.smithi153.stderr: 37950K .......... .......... .......... .......... .......... 51% 278M 0s 2019-11-25T09:29:35.890 INFO:teuthology.orchestra.run.smithi153.stderr: 38000K .......... .......... .......... .......... .......... 51% 327M 0s 2019-11-25T09:29:35.890 INFO:teuthology.orchestra.run.smithi153.stderr: 38050K .......... .......... .......... .......... .......... 51% 304M 0s 2019-11-25T09:29:35.891 INFO:teuthology.orchestra.run.smithi153.stderr: 38100K .......... .......... .......... .......... .......... 51% 299M 0s 2019-11-25T09:29:35.891 INFO:teuthology.orchestra.run.smithi153.stderr: 38150K .......... .......... .......... .......... .......... 51% 271M 0s 2019-11-25T09:29:35.891 INFO:teuthology.orchestra.run.smithi153.stderr: 38200K .......... .......... .......... .......... .......... 51% 285M 0s 2019-11-25T09:29:35.891 INFO:teuthology.orchestra.run.smithi153.stderr: 38250K .......... .......... .......... .......... .......... 52% 333M 0s 2019-11-25T09:29:35.891 INFO:teuthology.orchestra.run.smithi153.stderr: 38300K .......... .......... .......... .......... .......... 52% 339M 0s 2019-11-25T09:29:35.891 INFO:teuthology.orchestra.run.smithi153.stderr: 38350K .......... .......... .......... .......... .......... 52% 247M 0s 2019-11-25T09:29:35.891 INFO:teuthology.orchestra.run.smithi153.stderr: 38400K .......... .......... .......... .......... .......... 52% 291M 0s 2019-11-25T09:29:35.892 INFO:teuthology.orchestra.run.smithi153.stderr: 38450K .......... .......... .......... .......... .......... 52% 309M 0s 2019-11-25T09:29:35.892 INFO:teuthology.orchestra.run.smithi153.stderr: 38500K .......... .......... .......... .......... .......... 52% 341M 0s 2019-11-25T09:29:35.892 INFO:teuthology.orchestra.run.smithi153.stderr: 38550K .......... .......... .......... .......... .......... 52% 279M 0s 2019-11-25T09:29:35.892 INFO:teuthology.orchestra.run.smithi153.stderr: 38600K .......... .......... .......... .......... .......... 52% 318M 0s 2019-11-25T09:29:35.892 INFO:teuthology.orchestra.run.smithi153.stderr: 38650K .......... .......... .......... .......... .......... 52% 323M 0s 2019-11-25T09:29:35.892 INFO:teuthology.orchestra.run.smithi153.stderr: 38700K .......... .......... .......... .......... .......... 52% 297M 0s 2019-11-25T09:29:35.892 INFO:teuthology.orchestra.run.smithi153.stderr: 38750K .......... .......... .......... .......... .......... 52% 264M 0s 2019-11-25T09:29:35.893 INFO:teuthology.orchestra.run.smithi153.stderr: 38800K .......... .......... .......... .......... .......... 52% 317M 0s 2019-11-25T09:29:35.893 INFO:teuthology.orchestra.run.smithi153.stderr: 38850K .......... .......... .......... .......... .......... 52% 319M 0s 2019-11-25T09:29:35.893 INFO:teuthology.orchestra.run.smithi153.stderr: 38900K .......... .......... .......... .......... .......... 52% 348M 0s 2019-11-25T09:29:35.893 INFO:teuthology.orchestra.run.smithi153.stderr: 38950K .......... .......... .......... .......... .......... 53% 309M 0s 2019-11-25T09:29:35.893 INFO:teuthology.orchestra.run.smithi153.stderr: 39000K .......... .......... .......... .......... .......... 53% 344M 0s 2019-11-25T09:29:35.893 INFO:teuthology.orchestra.run.smithi153.stderr: 39050K .......... .......... .......... .......... .......... 53% 344M 0s 2019-11-25T09:29:35.893 INFO:teuthology.orchestra.run.smithi153.stderr: 39100K .......... .......... .......... .......... .......... 53% 359M 0s 2019-11-25T09:29:35.894 INFO:teuthology.orchestra.run.smithi153.stderr: 39150K .......... .......... .......... .......... .......... 53% 307M 0s 2019-11-25T09:29:35.894 INFO:teuthology.orchestra.run.smithi153.stderr: 39200K .......... .......... .......... .......... .......... 53% 348M 0s 2019-11-25T09:29:35.894 INFO:teuthology.orchestra.run.smithi153.stderr: 39250K .......... .......... .......... .......... .......... 53% 367M 0s 2019-11-25T09:29:35.894 INFO:teuthology.orchestra.run.smithi153.stderr: 39300K .......... .......... .......... .......... .......... 53% 350M 0s 2019-11-25T09:29:35.894 INFO:teuthology.orchestra.run.smithi153.stderr: 39350K .......... .......... .......... .......... .......... 53% 292M 0s 2019-11-25T09:29:35.894 INFO:teuthology.orchestra.run.smithi153.stderr: 39400K .......... .......... .......... .......... .......... 53% 358M 0s 2019-11-25T09:29:35.894 INFO:teuthology.orchestra.run.smithi153.stderr: 39450K .......... .......... .......... .......... .......... 53% 340M 0s 2019-11-25T09:29:35.895 INFO:teuthology.orchestra.run.smithi153.stderr: 39500K .......... .......... .......... .......... .......... 53% 338M 0s 2019-11-25T09:29:35.895 INFO:teuthology.orchestra.run.smithi153.stderr: 39550K .......... .......... .......... .......... .......... 53% 296M 0s 2019-11-25T09:29:35.895 INFO:teuthology.orchestra.run.smithi153.stderr: 39600K .......... .......... .......... .......... .......... 53% 320M 0s 2019-11-25T09:29:35.895 INFO:teuthology.orchestra.run.smithi153.stderr: 39650K .......... .......... .......... .......... .......... 53% 346M 0s 2019-11-25T09:29:35.895 INFO:teuthology.orchestra.run.smithi153.stderr: 39700K .......... .......... .......... .......... .......... 54% 334M 0s 2019-11-25T09:29:35.895 INFO:teuthology.orchestra.run.smithi153.stderr: 39750K .......... .......... .......... .......... .......... 54% 268M 0s 2019-11-25T09:29:35.895 INFO:teuthology.orchestra.run.smithi153.stderr: 39800K .......... .......... .......... .......... .......... 54% 357M 0s 2019-11-25T09:29:35.895 INFO:teuthology.orchestra.run.smithi153.stderr: 39850K .......... .......... .......... .......... .......... 54% 342M 0s 2019-11-25T09:29:35.896 INFO:teuthology.orchestra.run.smithi153.stderr: 39900K .......... .......... .......... .......... .......... 54% 349M 0s 2019-11-25T09:29:35.896 INFO:teuthology.orchestra.run.smithi153.stderr: 39950K .......... .......... .......... .......... .......... 54% 307M 0s 2019-11-25T09:29:35.896 INFO:teuthology.orchestra.run.smithi153.stderr: 40000K .......... .......... .......... .......... .......... 54% 339M 0s 2019-11-25T09:29:35.896 INFO:teuthology.orchestra.run.smithi153.stderr: 40050K .......... .......... .......... .......... .......... 54% 328M 0s 2019-11-25T09:29:35.896 INFO:teuthology.orchestra.run.smithi153.stderr: 40100K .......... .......... .......... .......... .......... 54% 324M 0s 2019-11-25T09:29:35.896 INFO:teuthology.orchestra.run.smithi153.stderr: 40150K .......... .......... .......... .......... .......... 54% 275M 0s 2019-11-25T09:29:35.897 INFO:teuthology.orchestra.run.smithi153.stderr: 40200K .......... .......... .......... .......... .......... 54% 299M 0s 2019-11-25T09:29:35.897 INFO:teuthology.orchestra.run.smithi153.stderr: 40250K .......... .......... .......... .......... .......... 54% 320M 0s 2019-11-25T09:29:35.897 INFO:teuthology.orchestra.run.smithi153.stderr: 40300K .......... .......... .......... .......... .......... 54% 317M 0s 2019-11-25T09:29:35.897 INFO:teuthology.orchestra.run.smithi153.stderr: 40350K .......... .......... .......... .......... .......... 54% 297M 0s 2019-11-25T09:29:35.897 INFO:teuthology.orchestra.run.smithi153.stderr: 40400K .......... .......... .......... .......... .......... 54% 333M 0s 2019-11-25T09:29:35.897 INFO:teuthology.orchestra.run.smithi153.stderr: 40450K .......... .......... .......... .......... .......... 55% 318M 0s 2019-11-25T09:29:35.897 INFO:teuthology.orchestra.run.smithi153.stderr: 40500K .......... .......... .......... .......... .......... 55% 323M 0s 2019-11-25T09:29:35.897 INFO:teuthology.orchestra.run.smithi153.stderr: 40550K .......... .......... .......... .......... .......... 55% 200M 0s 2019-11-25T09:29:35.898 INFO:teuthology.orchestra.run.smithi153.stderr: 40600K .......... .......... .......... .......... .......... 55% 261M 0s 2019-11-25T09:29:35.898 INFO:teuthology.orchestra.run.smithi153.stderr: 40650K .......... .......... .......... .......... .......... 55% 356M 0s 2019-11-25T09:29:35.898 INFO:teuthology.orchestra.run.smithi153.stderr: 40700K .......... .......... .......... .......... .......... 55% 302M 0s 2019-11-25T09:29:35.898 INFO:teuthology.orchestra.run.smithi153.stderr: 40750K .......... .......... .......... .......... .......... 55% 281M 0s 2019-11-25T09:29:35.898 INFO:teuthology.orchestra.run.smithi153.stderr: 40800K .......... .......... .......... .......... .......... 55% 346M 0s 2019-11-25T09:29:35.898 INFO:teuthology.orchestra.run.smithi153.stderr: 40850K .......... .......... .......... .......... .......... 55% 358M 0s 2019-11-25T09:29:35.898 INFO:teuthology.orchestra.run.smithi153.stderr: 40900K .......... .......... .......... .......... .......... 55% 352M 0s 2019-11-25T09:29:35.899 INFO:teuthology.orchestra.run.smithi153.stderr: 40950K .......... .......... .......... .......... .......... 55% 286M 0s 2019-11-25T09:29:35.899 INFO:teuthology.orchestra.run.smithi153.stderr: 41000K .......... .......... .......... .......... .......... 55% 346M 0s 2019-11-25T09:29:35.899 INFO:teuthology.orchestra.run.smithi153.stderr: 41050K .......... .......... .......... .......... .......... 55% 347M 0s 2019-11-25T09:29:35.899 INFO:teuthology.orchestra.run.smithi153.stderr: 41100K .......... .......... .......... .......... .......... 55% 364M 0s 2019-11-25T09:29:35.899 INFO:teuthology.orchestra.run.smithi153.stderr: 41150K .......... .......... .......... .......... .......... 56% 322M 0s 2019-11-25T09:29:35.899 INFO:teuthology.orchestra.run.smithi153.stderr: 41200K .......... .......... .......... .......... .......... 56% 349M 0s 2019-11-25T09:29:35.899 INFO:teuthology.orchestra.run.smithi153.stderr: 41250K .......... .......... .......... .......... .......... 56% 342M 0s 2019-11-25T09:29:35.900 INFO:teuthology.orchestra.run.smithi153.stderr: 41300K .......... .......... .......... .......... .......... 56% 331M 0s 2019-11-25T09:29:35.900 INFO:teuthology.orchestra.run.smithi153.stderr: 41350K .......... .......... .......... .......... .......... 56% 280M 0s 2019-11-25T09:29:35.900 INFO:teuthology.orchestra.run.smithi153.stderr: 41400K .......... .......... .......... .......... .......... 56% 341M 0s 2019-11-25T09:29:35.900 INFO:teuthology.orchestra.run.smithi153.stderr: 41450K .......... .......... .......... .......... .......... 56% 376M 0s 2019-11-25T09:29:35.900 INFO:teuthology.orchestra.run.smithi153.stderr: 41500K .......... .......... .......... .......... .......... 56% 339M 0s 2019-11-25T09:29:35.900 INFO:teuthology.orchestra.run.smithi153.stderr: 41550K .......... .......... .......... .......... .......... 56% 297M 0s 2019-11-25T09:29:35.900 INFO:teuthology.orchestra.run.smithi153.stderr: 41600K .......... .......... .......... .......... .......... 56% 335M 0s 2019-11-25T09:29:35.901 INFO:teuthology.orchestra.run.smithi153.stderr: 41650K .......... .......... .......... .......... .......... 56% 336M 0s 2019-11-25T09:29:35.901 INFO:teuthology.orchestra.run.smithi153.stderr: 41700K .......... .......... .......... .......... .......... 56% 376M 0s 2019-11-25T09:29:35.901 INFO:teuthology.orchestra.run.smithi153.stderr: 41750K .......... .......... .......... .......... .......... 56% 290M 0s 2019-11-25T09:29:35.901 INFO:teuthology.orchestra.run.smithi153.stderr: 41800K .......... .......... .......... .......... .......... 56% 337M 0s 2019-11-25T09:29:35.901 INFO:teuthology.orchestra.run.smithi153.stderr: 41850K .......... .......... .......... .......... .......... 56% 370M 0s 2019-11-25T09:29:35.901 INFO:teuthology.orchestra.run.smithi153.stderr: 41900K .......... .......... .......... .......... .......... 57% 366M 0s 2019-11-25T09:29:35.901 INFO:teuthology.orchestra.run.smithi153.stderr: 41950K .......... .......... .......... .......... .......... 57% 307M 0s 2019-11-25T09:29:35.902 INFO:teuthology.orchestra.run.smithi153.stderr: 42000K .......... .......... .......... .......... .......... 57% 350M 0s 2019-11-25T09:29:35.902 INFO:teuthology.orchestra.run.smithi153.stderr: 42050K .......... .......... .......... .......... .......... 57% 356M 0s 2019-11-25T09:29:35.902 INFO:teuthology.orchestra.run.smithi153.stderr: 42100K .......... .......... .......... .......... .......... 57% 362M 0s 2019-11-25T09:29:35.902 INFO:teuthology.orchestra.run.smithi153.stderr: 42150K .......... .......... .......... .......... .......... 57% 262M 0s 2019-11-25T09:29:35.902 INFO:teuthology.orchestra.run.smithi153.stderr: 42200K .......... .......... .......... .......... .......... 57% 327M 0s 2019-11-25T09:29:35.902 INFO:teuthology.orchestra.run.smithi153.stderr: 42250K .......... .......... .......... .......... .......... 57% 316M 0s 2019-11-25T09:29:35.902 INFO:teuthology.orchestra.run.smithi153.stderr: 42300K .......... .......... .......... .......... .......... 57% 315M 0s 2019-11-25T09:29:35.903 INFO:teuthology.orchestra.run.smithi153.stderr: 42350K .......... .......... .......... .......... .......... 57% 296M 0s 2019-11-25T09:29:35.903 INFO:teuthology.orchestra.run.smithi153.stderr: 42400K .......... .......... .......... .......... .......... 57% 353M 0s 2019-11-25T09:29:35.903 INFO:teuthology.orchestra.run.smithi153.stderr: 42450K .......... .......... .......... .......... .......... 57% 317M 0s 2019-11-25T09:29:35.903 INFO:teuthology.orchestra.run.smithi153.stderr: 42500K .......... .......... .......... .......... .......... 57% 347M 0s 2019-11-25T09:29:35.903 INFO:teuthology.orchestra.run.smithi153.stderr: 42550K .......... .......... .......... .......... .......... 57% 297M 0s 2019-11-25T09:29:35.903 INFO:teuthology.orchestra.run.smithi153.stderr: 42600K .......... .......... .......... .......... .......... 57% 344M 0s 2019-11-25T09:29:35.903 INFO:teuthology.orchestra.run.smithi153.stderr: 42650K .......... .......... .......... .......... .......... 58% 338M 0s 2019-11-25T09:29:35.903 INFO:teuthology.orchestra.run.smithi153.stderr: 42700K .......... .......... .......... .......... .......... 58% 369M 0s 2019-11-25T09:29:35.904 INFO:teuthology.orchestra.run.smithi153.stderr: 42750K .......... .......... .......... .......... .......... 58% 326M 0s 2019-11-25T09:29:35.904 INFO:teuthology.orchestra.run.smithi153.stderr: 42800K .......... .......... .......... .......... .......... 58% 356M 0s 2019-11-25T09:29:35.904 INFO:teuthology.orchestra.run.smithi153.stderr: 42850K .......... .......... .......... .......... .......... 58% 379M 0s 2019-11-25T09:29:35.904 INFO:teuthology.orchestra.run.smithi153.stderr: 42900K .......... .......... .......... .......... .......... 58% 365M 0s 2019-11-25T09:29:35.904 INFO:teuthology.orchestra.run.smithi153.stderr: 42950K .......... .......... .......... .......... .......... 58% 289M 0s 2019-11-25T09:29:35.904 INFO:teuthology.orchestra.run.smithi153.stderr: 43000K .......... .......... .......... .......... .......... 58% 297M 0s 2019-11-25T09:29:35.904 INFO:teuthology.orchestra.run.smithi153.stderr: 43050K .......... .......... .......... .......... .......... 58% 366M 0s 2019-11-25T09:29:35.905 INFO:teuthology.orchestra.run.smithi153.stderr: 43100K .......... .......... .......... .......... .......... 58% 363M 0s 2019-11-25T09:29:35.905 INFO:teuthology.orchestra.run.smithi153.stderr: 43150K .......... .......... .......... .......... .......... 58% 328M 0s 2019-11-25T09:29:35.905 INFO:teuthology.orchestra.run.smithi153.stderr: 43200K .......... .......... .......... .......... .......... 58% 364M 0s 2019-11-25T09:29:35.905 INFO:teuthology.orchestra.run.smithi153.stderr: 43250K .......... .......... .......... .......... .......... 58% 356M 0s 2019-11-25T09:29:35.905 INFO:teuthology.orchestra.run.smithi153.stderr: 43300K .......... .......... .......... .......... .......... 58% 351M 0s 2019-11-25T09:29:35.905 INFO:teuthology.orchestra.run.smithi153.stderr: 43350K .......... .......... .......... .......... .......... 59% 293M 0s 2019-11-25T09:29:35.905 INFO:teuthology.orchestra.run.smithi153.stderr: 43400K .......... .......... .......... .......... .......... 59% 362M 0s 2019-11-25T09:29:35.906 INFO:teuthology.orchestra.run.smithi153.stderr: 43450K .......... .......... .......... .......... .......... 59% 358M 0s 2019-11-25T09:29:35.906 INFO:teuthology.orchestra.run.smithi153.stderr: 43500K .......... .......... .......... .......... .......... 59% 347M 0s 2019-11-25T09:29:35.906 INFO:teuthology.orchestra.run.smithi153.stderr: 43550K .......... .......... .......... .......... .......... 59% 321M 0s 2019-11-25T09:29:35.906 INFO:teuthology.orchestra.run.smithi153.stderr: 43600K .......... .......... .......... .......... .......... 59% 353M 0s 2019-11-25T09:29:35.906 INFO:teuthology.orchestra.run.smithi153.stderr: 43650K .......... .......... .......... .......... .......... 59% 346M 0s 2019-11-25T09:29:35.906 INFO:teuthology.orchestra.run.smithi153.stderr: 43700K .......... .......... .......... .......... .......... 59% 367M 0s 2019-11-25T09:29:35.906 INFO:teuthology.orchestra.run.smithi153.stderr: 43750K .......... .......... .......... .......... .......... 59% 308M 0s 2019-11-25T09:29:35.907 INFO:teuthology.orchestra.run.smithi153.stderr: 43800K .......... .......... .......... .......... .......... 59% 353M 0s 2019-11-25T09:29:35.907 INFO:teuthology.orchestra.run.smithi153.stderr: 43850K .......... .......... .......... .......... .......... 59% 360M 0s 2019-11-25T09:29:35.907 INFO:teuthology.orchestra.run.smithi153.stderr: 43900K .......... .......... .......... .......... .......... 59% 372M 0s 2019-11-25T09:29:35.907 INFO:teuthology.orchestra.run.smithi153.stderr: 43950K .......... .......... .......... .......... .......... 59% 318M 0s 2019-11-25T09:29:35.907 INFO:teuthology.orchestra.run.smithi153.stderr: 44000K .......... .......... .......... .......... .......... 59% 370M 0s 2019-11-25T09:29:35.907 INFO:teuthology.orchestra.run.smithi153.stderr: 44050K .......... .......... .......... .......... .......... 59% 357M 0s 2019-11-25T09:29:35.907 INFO:teuthology.orchestra.run.smithi153.stderr: 44100K .......... .......... .......... .......... .......... 60% 371M 0s 2019-11-25T09:29:35.908 INFO:teuthology.orchestra.run.smithi153.stderr: 44150K .......... .......... .......... .......... .......... 60% 303M 0s 2019-11-25T09:29:35.908 INFO:teuthology.orchestra.run.smithi153.stderr: 44200K .......... .......... .......... .......... .......... 60% 361M 0s 2019-11-25T09:29:35.908 INFO:teuthology.orchestra.run.smithi153.stderr: 44250K .......... .......... .......... .......... .......... 60% 371M 0s 2019-11-25T09:29:35.908 INFO:teuthology.orchestra.run.smithi153.stderr: 44300K .......... .......... .......... .......... .......... 60% 367M 0s 2019-11-25T09:29:35.908 INFO:teuthology.orchestra.run.smithi153.stderr: 44350K .......... .......... .......... .......... .......... 60% 337M 0s 2019-11-25T09:29:35.908 INFO:teuthology.orchestra.run.smithi153.stderr: 44400K .......... .......... .......... .......... .......... 60% 352M 0s 2019-11-25T09:29:35.908 INFO:teuthology.orchestra.run.smithi153.stderr: 44450K .......... .......... .......... .......... .......... 60% 358M 0s 2019-11-25T09:29:35.909 INFO:teuthology.orchestra.run.smithi153.stderr: 44500K .......... .......... .......... .......... .......... 60% 384M 0s 2019-11-25T09:29:35.909 INFO:teuthology.orchestra.run.smithi153.stderr: 44550K .......... .......... .......... .......... .......... 60% 236M 0s 2019-11-25T09:29:35.909 INFO:teuthology.orchestra.run.smithi153.stderr: 44600K .......... .......... .......... .......... .......... 60% 299M 0s 2019-11-25T09:29:35.909 INFO:teuthology.orchestra.run.smithi153.stderr: 44650K .......... .......... .......... .......... .......... 60% 314M 0s 2019-11-25T09:29:35.909 INFO:teuthology.orchestra.run.smithi153.stderr: 44700K .......... .......... .......... .......... .......... 60% 308M 0s 2019-11-25T09:29:35.909 INFO:teuthology.orchestra.run.smithi153.stderr: 44750K .......... .......... .......... .......... .......... 60% 277M 0s 2019-11-25T09:29:35.909 INFO:teuthology.orchestra.run.smithi153.stderr: 44800K .......... .......... .......... .......... .......... 60% 316M 0s 2019-11-25T09:29:35.910 INFO:teuthology.orchestra.run.smithi153.stderr: 44850K .......... .......... .......... .......... .......... 61% 338M 0s 2019-11-25T09:29:35.911 INFO:teuthology.orchestra.run.smithi153.stderr: 44900K .......... .......... .......... .......... .......... 61% 385M 0s 2019-11-25T09:29:35.911 INFO:teuthology.orchestra.run.smithi153.stderr: 44950K .......... .......... .......... .......... .......... 61% 315M 0s 2019-11-25T09:29:35.911 INFO:teuthology.orchestra.run.smithi153.stderr: 45000K .......... .......... .......... .......... .......... 61% 386M 0s 2019-11-25T09:29:35.911 INFO:teuthology.orchestra.run.smithi153.stderr: 45050K .......... .......... .......... .......... .......... 61% 364M 0s 2019-11-25T09:29:35.912 INFO:teuthology.orchestra.run.smithi153.stderr: 45100K .......... .......... .......... .......... .......... 61% 369M 0s 2019-11-25T09:29:35.912 INFO:teuthology.orchestra.run.smithi153.stderr: 45150K .......... .......... .......... .......... .......... 61% 347M 0s 2019-11-25T09:29:35.912 INFO:teuthology.orchestra.run.smithi153.stderr: 45200K .......... .......... .......... .......... .......... 61% 374M 0s 2019-11-25T09:29:35.912 INFO:teuthology.orchestra.run.smithi153.stderr: 45250K .......... .......... .......... .......... .......... 61% 378M 0s 2019-11-25T09:29:35.912 INFO:teuthology.orchestra.run.smithi153.stderr: 45300K .......... .......... .......... .......... .......... 61% 358M 0s 2019-11-25T09:29:35.912 INFO:teuthology.orchestra.run.smithi153.stderr: 45350K .......... .......... .......... .......... .......... 61% 312M 0s 2019-11-25T09:29:35.912 INFO:teuthology.orchestra.run.smithi153.stderr: 45400K .......... .......... .......... .......... .......... 61% 367M 0s 2019-11-25T09:29:35.913 INFO:teuthology.orchestra.run.smithi153.stderr: 45450K .......... .......... .......... .......... .......... 61% 370M 0s 2019-11-25T09:29:35.913 INFO:teuthology.orchestra.run.smithi153.stderr: 45500K .......... .......... .......... .......... .......... 61% 385M 0s 2019-11-25T09:29:35.913 INFO:teuthology.orchestra.run.smithi153.stderr: 45550K .......... .......... .......... .......... .......... 61% 282M 0s 2019-11-25T09:29:35.913 INFO:teuthology.orchestra.run.smithi153.stderr: 45600K .......... .......... .......... .......... .......... 62% 338M 0s 2019-11-25T09:29:35.913 INFO:teuthology.orchestra.run.smithi153.stderr: 45650K .......... .......... .......... .......... .......... 62% 361M 0s 2019-11-25T09:29:35.913 INFO:teuthology.orchestra.run.smithi153.stderr: 45700K .......... .......... .......... .......... .......... 62% 341M 0s 2019-11-25T09:29:35.913 INFO:teuthology.orchestra.run.smithi153.stderr: 45750K .......... .......... .......... .......... .......... 62% 325M 0s 2019-11-25T09:29:35.914 INFO:teuthology.orchestra.run.smithi153.stderr: 45800K .......... .......... .......... .......... .......... 62% 338M 0s 2019-11-25T09:29:35.914 INFO:teuthology.orchestra.run.smithi153.stderr: 45850K .......... .......... .......... .......... .......... 62% 346M 0s 2019-11-25T09:29:35.914 INFO:teuthology.orchestra.run.smithi153.stderr: 45900K .......... .......... .......... .......... .......... 62% 342M 0s 2019-11-25T09:29:35.914 INFO:teuthology.orchestra.run.smithi153.stderr: 45950K .......... .......... .......... .......... .......... 62% 287M 0s 2019-11-25T09:29:35.914 INFO:teuthology.orchestra.run.smithi153.stderr: 46000K .......... .......... .......... .......... .......... 62% 374M 0s 2019-11-25T09:29:35.914 INFO:teuthology.orchestra.run.smithi153.stderr: 46050K .......... .......... .......... .......... .......... 62% 379M 0s 2019-11-25T09:29:35.914 INFO:teuthology.orchestra.run.smithi153.stderr: 46100K .......... .......... .......... .......... .......... 62% 359M 0s 2019-11-25T09:29:35.915 INFO:teuthology.orchestra.run.smithi153.stderr: 46150K .......... .......... .......... .......... .......... 62% 318M 0s 2019-11-25T09:29:35.915 INFO:teuthology.orchestra.run.smithi153.stderr: 46200K .......... .......... .......... .......... .......... 62% 354M 0s 2019-11-25T09:29:35.915 INFO:teuthology.orchestra.run.smithi153.stderr: 46250K .......... .......... .......... .......... .......... 62% 360M 0s 2019-11-25T09:29:35.915 INFO:teuthology.orchestra.run.smithi153.stderr: 46300K .......... .......... .......... .......... .......... 63% 358M 0s 2019-11-25T09:29:35.915 INFO:teuthology.orchestra.run.smithi153.stderr: 46350K .......... .......... .......... .......... .......... 63% 339M 0s 2019-11-25T09:29:35.915 INFO:teuthology.orchestra.run.smithi153.stderr: 46400K .......... .......... .......... .......... .......... 63% 356M 0s 2019-11-25T09:29:35.915 INFO:teuthology.orchestra.run.smithi153.stderr: 46450K .......... .......... .......... .......... .......... 63% 374M 0s 2019-11-25T09:29:35.916 INFO:teuthology.orchestra.run.smithi153.stderr: 46500K .......... .......... .......... .......... .......... 63% 377M 0s 2019-11-25T09:29:35.916 INFO:teuthology.orchestra.run.smithi153.stderr: 46550K .......... .......... .......... .......... .......... 63% 325M 0s 2019-11-25T09:29:35.916 INFO:teuthology.orchestra.run.smithi153.stderr: 46600K .......... .......... .......... .......... .......... 63% 376M 0s 2019-11-25T09:29:35.916 INFO:teuthology.orchestra.run.smithi153.stderr: 46650K .......... .......... .......... .......... .......... 63% 378M 0s 2019-11-25T09:29:35.916 INFO:teuthology.orchestra.run.smithi153.stderr: 46700K .......... .......... .......... .......... .......... 63% 370M 0s 2019-11-25T09:29:35.916 INFO:teuthology.orchestra.run.smithi153.stderr: 46750K .......... .......... .......... .......... .......... 63% 332M 0s 2019-11-25T09:29:35.916 INFO:teuthology.orchestra.run.smithi153.stderr: 46800K .......... .......... .......... .......... .......... 63% 351M 0s 2019-11-25T09:29:35.917 INFO:teuthology.orchestra.run.smithi153.stderr: 46850K .......... .......... .......... .......... .......... 63% 322M 0s 2019-11-25T09:29:35.917 INFO:teuthology.orchestra.run.smithi153.stderr: 46900K .......... .......... .......... .......... .......... 63% 365M 0s 2019-11-25T09:29:35.917 INFO:teuthology.orchestra.run.smithi153.stderr: 46950K .......... .......... .......... .......... .......... 63% 319M 0s 2019-11-25T09:29:35.917 INFO:teuthology.orchestra.run.smithi153.stderr: 47000K .......... .......... .......... .......... .......... 63% 370M 0s 2019-11-25T09:29:35.917 INFO:teuthology.orchestra.run.smithi153.stderr: 47050K .......... .......... .......... .......... .......... 64% 371M 0s 2019-11-25T09:29:35.917 INFO:teuthology.orchestra.run.smithi153.stderr: 47100K .......... .......... .......... .......... .......... 64% 353M 0s 2019-11-25T09:29:35.917 INFO:teuthology.orchestra.run.smithi153.stderr: 47150K .......... .......... .......... .......... .......... 64% 297M 0s 2019-11-25T09:29:35.918 INFO:teuthology.orchestra.run.smithi153.stderr: 47200K .......... .......... .......... .......... .......... 64% 363M 0s 2019-11-25T09:29:35.918 INFO:teuthology.orchestra.run.smithi153.stderr: 47250K .......... .......... .......... .......... .......... 64% 370M 0s 2019-11-25T09:29:35.918 INFO:teuthology.orchestra.run.smithi153.stderr: 47300K .......... .......... .......... .......... .......... 64% 311M 0s 2019-11-25T09:29:35.918 INFO:teuthology.orchestra.run.smithi153.stderr: 47350K .......... .......... .......... .......... .......... 64% 315M 0s 2019-11-25T09:29:35.918 INFO:teuthology.orchestra.run.smithi153.stderr: 47400K .......... .......... .......... .......... .......... 64% 334M 0s 2019-11-25T09:29:35.918 INFO:teuthology.orchestra.run.smithi153.stderr: 47450K .......... .......... .......... .......... .......... 64% 346M 0s 2019-11-25T09:29:35.918 INFO:teuthology.orchestra.run.smithi153.stderr: 47500K .......... .......... .......... .......... .......... 64% 350M 0s 2019-11-25T09:29:35.919 INFO:teuthology.orchestra.run.smithi153.stderr: 47550K .......... .......... .......... .......... .......... 64% 285M 0s 2019-11-25T09:29:35.919 INFO:teuthology.orchestra.run.smithi153.stderr: 47600K .......... .......... .......... .......... .......... 64% 373M 0s 2019-11-25T09:29:35.919 INFO:teuthology.orchestra.run.smithi153.stderr: 47650K .......... .......... .......... .......... .......... 64% 359M 0s 2019-11-25T09:29:35.919 INFO:teuthology.orchestra.run.smithi153.stderr: 47700K .......... .......... .......... .......... .......... 64% 355M 0s 2019-11-25T09:29:35.919 INFO:teuthology.orchestra.run.smithi153.stderr: 47750K .......... .......... .......... .......... .......... 64% 315M 0s 2019-11-25T09:29:35.919 INFO:teuthology.orchestra.run.smithi153.stderr: 47800K .......... .......... .......... .......... .......... 65% 376M 0s 2019-11-25T09:29:35.919 INFO:teuthology.orchestra.run.smithi153.stderr: 47850K .......... .......... .......... .......... .......... 65% 367M 0s 2019-11-25T09:29:35.919 INFO:teuthology.orchestra.run.smithi153.stderr: 47900K .......... .......... .......... .......... .......... 65% 351M 0s 2019-11-25T09:29:35.920 INFO:teuthology.orchestra.run.smithi153.stderr: 47950K .......... .......... .......... .......... .......... 65% 324M 0s 2019-11-25T09:29:35.920 INFO:teuthology.orchestra.run.smithi153.stderr: 48000K .......... .......... .......... .......... .......... 65% 366M 0s 2019-11-25T09:29:35.920 INFO:teuthology.orchestra.run.smithi153.stderr: 48050K .......... .......... .......... .......... .......... 65% 343M 0s 2019-11-25T09:29:35.920 INFO:teuthology.orchestra.run.smithi153.stderr: 48100K .......... .......... .......... .......... .......... 65% 320M 0s 2019-11-25T09:29:35.920 INFO:teuthology.orchestra.run.smithi153.stderr: 48150K .......... .......... .......... .......... .......... 65% 320M 0s 2019-11-25T09:29:35.920 INFO:teuthology.orchestra.run.smithi153.stderr: 48200K .......... .......... .......... .......... .......... 65% 350M 0s 2019-11-25T09:29:35.920 INFO:teuthology.orchestra.run.smithi153.stderr: 48250K .......... .......... .......... .......... .......... 65% 300M 0s 2019-11-25T09:29:35.921 INFO:teuthology.orchestra.run.smithi153.stderr: 48300K .......... .......... .......... .......... .......... 65% 328M 0s 2019-11-25T09:29:35.921 INFO:teuthology.orchestra.run.smithi153.stderr: 48350K .......... .......... .......... .......... .......... 65% 275M 0s 2019-11-25T09:29:35.921 INFO:teuthology.orchestra.run.smithi153.stderr: 48400K .......... .......... .......... .......... .......... 65% 329M 0s 2019-11-25T09:29:35.921 INFO:teuthology.orchestra.run.smithi153.stderr: 48450K .......... .......... .......... .......... .......... 65% 375M 0s 2019-11-25T09:29:35.921 INFO:teuthology.orchestra.run.smithi153.stderr: 48500K .......... .......... .......... .......... .......... 66% 338M 0s 2019-11-25T09:29:35.921 INFO:teuthology.orchestra.run.smithi153.stderr: 48550K .......... .......... .......... .......... .......... 66% 289M 0s 2019-11-25T09:29:35.921 INFO:teuthology.orchestra.run.smithi153.stderr: 48600K .......... .......... .......... .......... .......... 66% 273M 0s 2019-11-25T09:29:35.922 INFO:teuthology.orchestra.run.smithi153.stderr: 48650K .......... .......... .......... .......... .......... 66% 294M 0s 2019-11-25T09:29:35.922 INFO:teuthology.orchestra.run.smithi153.stderr: 48700K .......... .......... .......... .......... .......... 66% 292M 0s 2019-11-25T09:29:35.922 INFO:teuthology.orchestra.run.smithi153.stderr: 48750K .......... .......... .......... .......... .......... 66% 238M 0s 2019-11-25T09:29:35.922 INFO:teuthology.orchestra.run.smithi153.stderr: 48800K .......... .......... .......... .......... .......... 66% 291M 0s 2019-11-25T09:29:35.922 INFO:teuthology.orchestra.run.smithi153.stderr: 48850K .......... .......... .......... .......... .......... 66% 301M 0s 2019-11-25T09:29:35.922 INFO:teuthology.orchestra.run.smithi153.stderr: 48900K .......... .......... .......... .......... .......... 66% 316M 0s 2019-11-25T09:29:35.922 INFO:teuthology.orchestra.run.smithi153.stderr: 48950K .......... .......... .......... .......... .......... 66% 241M 0s 2019-11-25T09:29:35.923 INFO:teuthology.orchestra.run.smithi153.stderr: 49000K .......... .......... .......... .......... .......... 66% 280M 0s 2019-11-25T09:29:35.923 INFO:teuthology.orchestra.run.smithi153.stderr: 49050K .......... .......... .......... .......... .......... 66% 303M 0s 2019-11-25T09:29:35.923 INFO:teuthology.orchestra.run.smithi153.stderr: 49100K .......... .......... .......... .......... .......... 66% 289M 0s 2019-11-25T09:29:35.923 INFO:teuthology.orchestra.run.smithi153.stderr: 49150K .......... .......... .......... .......... .......... 66% 251M 0s 2019-11-25T09:29:35.923 INFO:teuthology.orchestra.run.smithi153.stderr: 49200K .......... .......... .......... .......... .......... 66% 279M 0s 2019-11-25T09:29:35.923 INFO:teuthology.orchestra.run.smithi153.stderr: 49250K .......... .......... .......... .......... .......... 67% 289M 0s 2019-11-25T09:29:35.923 INFO:teuthology.orchestra.run.smithi153.stderr: 49300K .......... .......... .......... .......... .......... 67% 312M 0s 2019-11-25T09:29:35.924 INFO:teuthology.orchestra.run.smithi153.stderr: 49350K .......... .......... .......... .......... .......... 67% 254M 0s 2019-11-25T09:29:35.924 INFO:teuthology.orchestra.run.smithi153.stderr: 49400K .......... .......... .......... .......... .......... 67% 300M 0s 2019-11-25T09:29:35.924 INFO:teuthology.orchestra.run.smithi153.stderr: 49450K .......... .......... .......... .......... .......... 67% 296M 0s 2019-11-25T09:29:35.924 INFO:teuthology.orchestra.run.smithi153.stderr: 49500K .......... .......... .......... .......... .......... 67% 284M 0s 2019-11-25T09:29:35.924 INFO:teuthology.orchestra.run.smithi153.stderr: 49550K .......... .......... .......... .......... .......... 67% 276M 0s 2019-11-25T09:29:35.924 INFO:teuthology.orchestra.run.smithi153.stderr: 49600K .......... .......... .......... .......... .......... 67% 304M 0s 2019-11-25T09:29:35.924 INFO:teuthology.orchestra.run.smithi153.stderr: 49650K .......... .......... .......... .......... .......... 67% 301M 0s 2019-11-25T09:29:35.925 INFO:teuthology.orchestra.run.smithi153.stderr: 49700K .......... .......... .......... .......... .......... 67% 307M 0s 2019-11-25T09:29:35.925 INFO:teuthology.orchestra.run.smithi153.stderr: 49750K .......... .......... .......... .......... .......... 67% 243M 0s 2019-11-25T09:29:35.925 INFO:teuthology.orchestra.run.smithi153.stderr: 49800K .......... .......... .......... .......... .......... 67% 298M 0s 2019-11-25T09:29:35.925 INFO:teuthology.orchestra.run.smithi153.stderr: 49850K .......... .......... .......... .......... .......... 67% 306M 0s 2019-11-25T09:29:35.925 INFO:teuthology.orchestra.run.smithi153.stderr: 49900K .......... .......... .......... .......... .......... 67% 314M 0s 2019-11-25T09:29:35.925 INFO:teuthology.orchestra.run.smithi153.stderr: 49950K .......... .......... .......... .......... .......... 67% 245M 0s 2019-11-25T09:29:35.925 INFO:teuthology.orchestra.run.smithi153.stderr: 50000K .......... .......... .......... .......... .......... 68% 313M 0s 2019-11-25T09:29:35.926 INFO:teuthology.orchestra.run.smithi153.stderr: 50050K .......... .......... .......... .......... .......... 68% 305M 0s 2019-11-25T09:29:35.926 INFO:teuthology.orchestra.run.smithi153.stderr: 50100K .......... .......... .......... .......... .......... 68% 305M 0s 2019-11-25T09:29:35.926 INFO:teuthology.orchestra.run.smithi153.stderr: 50150K .......... .......... .......... .......... .......... 68% 264M 0s 2019-11-25T09:29:35.926 INFO:teuthology.orchestra.run.smithi153.stderr: 50200K .......... .......... .......... .......... .......... 68% 292M 0s 2019-11-25T09:29:35.926 INFO:teuthology.orchestra.run.smithi153.stderr: 50250K .......... .......... .......... .......... .......... 68% 297M 0s 2019-11-25T09:29:35.926 INFO:teuthology.orchestra.run.smithi153.stderr: 50300K .......... .......... .......... .......... .......... 68% 286M 0s 2019-11-25T09:29:35.926 INFO:teuthology.orchestra.run.smithi153.stderr: 50350K .......... .......... .......... .......... .......... 68% 228M 0s 2019-11-25T09:29:35.927 INFO:teuthology.orchestra.run.smithi153.stderr: 50400K .......... .......... .......... .......... .......... 68% 310M 0s 2019-11-25T09:29:35.927 INFO:teuthology.orchestra.run.smithi153.stderr: 50450K .......... .......... .......... .......... .......... 68% 308M 0s 2019-11-25T09:29:35.927 INFO:teuthology.orchestra.run.smithi153.stderr: 50500K .......... .......... .......... .......... .......... 68% 336M 0s 2019-11-25T09:29:35.927 INFO:teuthology.orchestra.run.smithi153.stderr: 50550K .......... .......... .......... .......... .......... 68% 307M 0s 2019-11-25T09:29:35.927 INFO:teuthology.orchestra.run.smithi153.stderr: 50600K .......... .......... .......... .......... .......... 68% 349M 0s 2019-11-25T09:29:35.927 INFO:teuthology.orchestra.run.smithi153.stderr: 50650K .......... .......... .......... .......... .......... 68% 324M 0s 2019-11-25T09:29:35.927 INFO:teuthology.orchestra.run.smithi153.stderr: 50700K .......... .......... .......... .......... .......... 68% 340M 0s 2019-11-25T09:29:35.927 INFO:teuthology.orchestra.run.smithi153.stderr: 50750K .......... .......... .......... .......... .......... 69% 300M 0s 2019-11-25T09:29:35.928 INFO:teuthology.orchestra.run.smithi153.stderr: 50800K .......... .......... .......... .......... .......... 69% 358M 0s 2019-11-25T09:29:35.928 INFO:teuthology.orchestra.run.smithi153.stderr: 50850K .......... .......... .......... .......... .......... 69% 360M 0s 2019-11-25T09:29:35.928 INFO:teuthology.orchestra.run.smithi153.stderr: 50900K .......... .......... .......... .......... .......... 69% 321M 0s 2019-11-25T09:29:35.928 INFO:teuthology.orchestra.run.smithi153.stderr: 50950K .......... .......... .......... .......... .......... 69% 322M 0s 2019-11-25T09:29:35.928 INFO:teuthology.orchestra.run.smithi153.stderr: 51000K .......... .......... .......... .......... .......... 69% 358M 0s 2019-11-25T09:29:35.928 INFO:teuthology.orchestra.run.smithi153.stderr: 51050K .......... .......... .......... .......... .......... 69% 360M 0s 2019-11-25T09:29:35.928 INFO:teuthology.orchestra.run.smithi153.stderr: 51100K .......... .......... .......... .......... .......... 69% 330M 0s 2019-11-25T09:29:35.929 INFO:teuthology.orchestra.run.smithi153.stderr: 51150K .......... .......... .......... .......... .......... 69% 329M 0s 2019-11-25T09:29:35.929 INFO:teuthology.orchestra.run.smithi153.stderr: 51200K .......... .......... .......... .......... .......... 69% 382M 0s 2019-11-25T09:29:35.929 INFO:teuthology.orchestra.run.smithi153.stderr: 51250K .......... .......... .......... .......... .......... 69% 356M 0s 2019-11-25T09:29:35.929 INFO:teuthology.orchestra.run.smithi153.stderr: 51300K .......... .......... .......... .......... .......... 69% 365M 0s 2019-11-25T09:29:35.929 INFO:teuthology.orchestra.run.smithi153.stderr: 51350K .......... .......... .......... .......... .......... 69% 304M 0s 2019-11-25T09:29:35.929 INFO:teuthology.orchestra.run.smithi153.stderr: 51400K .......... .......... .......... .......... .......... 69% 379M 0s 2019-11-25T09:29:35.929 INFO:teuthology.orchestra.run.smithi153.stderr: 51450K .......... .......... .......... .......... .......... 70% 388M 0s 2019-11-25T09:29:35.930 INFO:teuthology.orchestra.run.smithi153.stderr: 51500K .......... .......... .......... .......... .......... 70% 346M 0s 2019-11-25T09:29:35.930 INFO:teuthology.orchestra.run.smithi153.stderr: 51550K .......... .......... .......... .......... .......... 70% 326M 0s 2019-11-25T09:29:35.930 INFO:teuthology.orchestra.run.smithi153.stderr: 51600K .......... .......... .......... .......... .......... 70% 385M 0s 2019-11-25T09:29:35.930 INFO:teuthology.orchestra.run.smithi153.stderr: 51650K .......... .......... .......... .......... .......... 70% 357M 0s 2019-11-25T09:29:35.930 INFO:teuthology.orchestra.run.smithi153.stderr: 51700K .......... .......... .......... .......... .......... 70% 356M 0s 2019-11-25T09:29:35.930 INFO:teuthology.orchestra.run.smithi153.stderr: 51750K .......... .......... .......... .......... .......... 70% 319M 0s 2019-11-25T09:29:35.930 INFO:teuthology.orchestra.run.smithi153.stderr: 51800K .......... .......... .......... .......... .......... 70% 371M 0s 2019-11-25T09:29:35.931 INFO:teuthology.orchestra.run.smithi153.stderr: 51850K .......... .......... .......... .......... .......... 70% 364M 0s 2019-11-25T09:29:35.931 INFO:teuthology.orchestra.run.smithi153.stderr: 51900K .......... .......... .......... .......... .......... 70% 353M 0s 2019-11-25T09:29:35.931 INFO:teuthology.orchestra.run.smithi153.stderr: 51950K .......... .......... .......... .......... .......... 70% 324M 0s 2019-11-25T09:29:35.931 INFO:teuthology.orchestra.run.smithi153.stderr: 52000K .......... .......... .......... .......... .......... 70% 350M 0s 2019-11-25T09:29:35.931 INFO:teuthology.orchestra.run.smithi153.stderr: 52050K .......... .......... .......... .......... .......... 70% 331M 0s 2019-11-25T09:29:35.931 INFO:teuthology.orchestra.run.smithi153.stderr: 52100K .......... .......... .......... .......... .......... 70% 371M 0s 2019-11-25T09:29:35.931 INFO:teuthology.orchestra.run.smithi153.stderr: 52150K .......... .......... .......... .......... .......... 70% 341M 0s 2019-11-25T09:29:35.932 INFO:teuthology.orchestra.run.smithi153.stderr: 52200K .......... .......... .......... .......... .......... 71% 363M 0s 2019-11-25T09:29:35.932 INFO:teuthology.orchestra.run.smithi153.stderr: 52250K .......... .......... .......... .......... .......... 71% 374M 0s 2019-11-25T09:29:35.932 INFO:teuthology.orchestra.run.smithi153.stderr: 52300K .......... .......... .......... .......... .......... 71% 370M 0s 2019-11-25T09:29:35.932 INFO:teuthology.orchestra.run.smithi153.stderr: 52350K .......... .......... .......... .......... .......... 71% 293M 0s 2019-11-25T09:29:35.932 INFO:teuthology.orchestra.run.smithi153.stderr: 52400K .......... .......... .......... .......... .......... 71% 327M 0s 2019-11-25T09:29:35.932 INFO:teuthology.orchestra.run.smithi153.stderr: 52450K .......... .......... .......... .......... .......... 71% 377M 0s 2019-11-25T09:29:35.932 INFO:teuthology.orchestra.run.smithi153.stderr: 52500K .......... .......... .......... .......... .......... 71% 398M 0s 2019-11-25T09:29:35.933 INFO:teuthology.orchestra.run.smithi153.stderr: 52550K .......... .......... .......... .......... .......... 71% 302M 0s 2019-11-25T09:29:35.933 INFO:teuthology.orchestra.run.smithi153.stderr: 52600K .......... .......... .......... .......... .......... 71% 385M 0s 2019-11-25T09:29:35.933 INFO:teuthology.orchestra.run.smithi153.stderr: 52650K .......... .......... .......... .......... .......... 71% 355M 0s 2019-11-25T09:29:35.933 INFO:teuthology.orchestra.run.smithi153.stderr: 52700K .......... .......... .......... .......... .......... 71% 389M 0s 2019-11-25T09:29:35.933 INFO:teuthology.orchestra.run.smithi153.stderr: 52750K .......... .......... .......... .......... .......... 71% 317M 0s 2019-11-25T09:29:35.933 INFO:teuthology.orchestra.run.smithi153.stderr: 52800K .......... .......... .......... .......... .......... 71% 399M 0s 2019-11-25T09:29:35.933 INFO:teuthology.orchestra.run.smithi153.stderr: 52850K .......... .......... .......... .......... .......... 71% 400M 0s 2019-11-25T09:29:35.934 INFO:teuthology.orchestra.run.smithi153.stderr: 52900K .......... .......... .......... .......... .......... 71% 408M 0s 2019-11-25T09:29:35.934 INFO:teuthology.orchestra.run.smithi153.stderr: 52950K .......... .......... .......... .......... .......... 72% 326M 0s 2019-11-25T09:29:35.934 INFO:teuthology.orchestra.run.smithi153.stderr: 53000K .......... .......... .......... .......... .......... 72% 402M 0s 2019-11-25T09:29:35.934 INFO:teuthology.orchestra.run.smithi153.stderr: 53050K .......... .......... .......... .......... .......... 72% 382M 0s 2019-11-25T09:29:35.934 INFO:teuthology.orchestra.run.smithi153.stderr: 53100K .......... .......... .......... .......... .......... 72% 398M 0s 2019-11-25T09:29:35.934 INFO:teuthology.orchestra.run.smithi153.stderr: 53150K .......... .......... .......... .......... .......... 72% 333M 0s 2019-11-25T09:29:35.934 INFO:teuthology.orchestra.run.smithi153.stderr: 53200K .......... .......... .......... .......... .......... 72% 381M 0s 2019-11-25T09:29:35.935 INFO:teuthology.orchestra.run.smithi153.stderr: 53250K .......... .......... .......... .......... .......... 72% 401M 0s 2019-11-25T09:29:35.935 INFO:teuthology.orchestra.run.smithi153.stderr: 53300K .......... .......... .......... .......... .......... 72% 384M 0s 2019-11-25T09:29:35.935 INFO:teuthology.orchestra.run.smithi153.stderr: 53350K .......... .......... .......... .......... .......... 72% 341M 0s 2019-11-25T09:29:35.935 INFO:teuthology.orchestra.run.smithi153.stderr: 53400K .......... .......... .......... .......... .......... 72% 405M 0s 2019-11-25T09:29:35.935 INFO:teuthology.orchestra.run.smithi153.stderr: 53450K .......... .......... .......... .......... .......... 72% 379M 0s 2019-11-25T09:29:35.935 INFO:teuthology.orchestra.run.smithi153.stderr: 53500K .......... .......... .......... .......... .......... 72% 380M 0s 2019-11-25T09:29:35.935 INFO:teuthology.orchestra.run.smithi153.stderr: 53550K .......... .......... .......... .......... .......... 72% 315M 0s 2019-11-25T09:29:35.935 INFO:teuthology.orchestra.run.smithi153.stderr: 53600K .......... .......... .......... .......... .......... 72% 401M 0s 2019-11-25T09:29:35.936 INFO:teuthology.orchestra.run.smithi153.stderr: 53650K .......... .......... .......... .......... .......... 73% 380M 0s 2019-11-25T09:29:35.936 INFO:teuthology.orchestra.run.smithi153.stderr: 53700K .......... .......... .......... .......... .......... 73% 394M 0s 2019-11-25T09:29:35.936 INFO:teuthology.orchestra.run.smithi153.stderr: 53750K .......... .......... .......... .......... .......... 73% 343M 0s 2019-11-25T09:29:35.936 INFO:teuthology.orchestra.run.smithi153.stderr: 53800K .......... .......... .......... .......... .......... 73% 360M 0s 2019-11-25T09:29:35.936 INFO:teuthology.orchestra.run.smithi153.stderr: 53850K .......... .......... .......... .......... .......... 73% 386M 0s 2019-11-25T09:29:35.936 INFO:teuthology.orchestra.run.smithi153.stderr: 53900K .......... .......... .......... .......... .......... 73% 377M 0s 2019-11-25T09:29:35.936 INFO:teuthology.orchestra.run.smithi153.stderr: 53950K .......... .......... .......... .......... .......... 73% 319M 0s 2019-11-25T09:29:35.937 INFO:teuthology.orchestra.run.smithi153.stderr: 54000K .......... .......... .......... .......... .......... 73% 385M 0s 2019-11-25T09:29:35.937 INFO:teuthology.orchestra.run.smithi153.stderr: 54050K .......... .......... .......... .......... .......... 73% 385M 0s 2019-11-25T09:29:35.937 INFO:teuthology.orchestra.run.smithi153.stderr: 54100K .......... .......... .......... .......... .......... 73% 380M 0s 2019-11-25T09:29:35.937 INFO:teuthology.orchestra.run.smithi153.stderr: 54150K .......... .......... .......... .......... .......... 73% 370M 0s 2019-11-25T09:29:35.937 INFO:teuthology.orchestra.run.smithi153.stderr: 54200K .......... .......... .......... .......... .......... 73% 389M 0s 2019-11-25T09:29:35.937 INFO:teuthology.orchestra.run.smithi153.stderr: 54250K .......... .......... .......... .......... .......... 73% 356M 0s 2019-11-25T09:29:35.937 INFO:teuthology.orchestra.run.smithi153.stderr: 54300K .......... .......... .......... .......... .......... 73% 360M 0s 2019-11-25T09:29:35.938 INFO:teuthology.orchestra.run.smithi153.stderr: 54350K .......... .......... .......... .......... .......... 73% 307M 0s 2019-11-25T09:29:35.938 INFO:teuthology.orchestra.run.smithi153.stderr: 54400K .......... .......... .......... .......... .......... 74% 360M 0s 2019-11-25T09:29:35.938 INFO:teuthology.orchestra.run.smithi153.stderr: 54450K .......... .......... .......... .......... .......... 74% 364M 0s 2019-11-25T09:29:35.938 INFO:teuthology.orchestra.run.smithi153.stderr: 54500K .......... .......... .......... .......... .......... 74% 388M 0s 2019-11-25T09:29:35.938 INFO:teuthology.orchestra.run.smithi153.stderr: 54550K .......... .......... .......... .......... .......... 74% 317M 0s 2019-11-25T09:29:35.938 INFO:teuthology.orchestra.run.smithi153.stderr: 54600K .......... .......... .......... .......... .......... 74% 361M 0s 2019-11-25T09:29:35.938 INFO:teuthology.orchestra.run.smithi153.stderr: 54650K .......... .......... .......... .......... .......... 74% 358M 0s 2019-11-25T09:29:35.939 INFO:teuthology.orchestra.run.smithi153.stderr: 54700K .......... .......... .......... .......... .......... 74% 409M 0s 2019-11-25T09:29:35.939 INFO:teuthology.orchestra.run.smithi153.stderr: 54750K .......... .......... .......... .......... .......... 74% 320M 0s 2019-11-25T09:29:35.939 INFO:teuthology.orchestra.run.smithi153.stderr: 54800K .......... .......... .......... .......... .......... 74% 379M 0s 2019-11-25T09:29:35.939 INFO:teuthology.orchestra.run.smithi153.stderr: 54850K .......... .......... .......... .......... .......... 74% 397M 0s 2019-11-25T09:29:35.939 INFO:teuthology.orchestra.run.smithi153.stderr: 54900K .......... .......... .......... .......... .......... 74% 389M 0s 2019-11-25T09:29:35.939 INFO:teuthology.orchestra.run.smithi153.stderr: 54950K .......... .......... .......... .......... .......... 74% 344M 0s 2019-11-25T09:29:35.939 INFO:teuthology.orchestra.run.smithi153.stderr: 55000K .......... .......... .......... .......... .......... 74% 387M 0s 2019-11-25T09:29:35.940 INFO:teuthology.orchestra.run.smithi153.stderr: 55050K .......... .......... .......... .......... .......... 74% 384M 0s 2019-11-25T09:29:35.940 INFO:teuthology.orchestra.run.smithi153.stderr: 55100K .......... .......... .......... .......... .......... 74% 408M 0s 2019-11-25T09:29:35.940 INFO:teuthology.orchestra.run.smithi153.stderr: 55150K .......... .......... .......... .......... .......... 75% 318M 0s 2019-11-25T09:29:35.940 INFO:teuthology.orchestra.run.smithi153.stderr: 55200K .......... .......... .......... .......... .......... 75% 377M 0s 2019-11-25T09:29:35.940 INFO:teuthology.orchestra.run.smithi153.stderr: 55250K .......... .......... .......... .......... .......... 75% 366M 0s 2019-11-25T09:29:35.940 INFO:teuthology.orchestra.run.smithi153.stderr: 55300K .......... .......... .......... .......... .......... 75% 389M 0s 2019-11-25T09:29:35.940 INFO:teuthology.orchestra.run.smithi153.stderr: 55350K .......... .......... .......... .......... .......... 75% 353M 0s 2019-11-25T09:29:35.941 INFO:teuthology.orchestra.run.smithi153.stderr: 55400K .......... .......... .......... .......... .......... 75% 377M 0s 2019-11-25T09:29:35.941 INFO:teuthology.orchestra.run.smithi153.stderr: 55450K .......... .......... .......... .......... .......... 75% 383M 0s 2019-11-25T09:29:35.941 INFO:teuthology.orchestra.run.smithi153.stderr: 55500K .......... .......... .......... .......... .......... 75% 384M 0s 2019-11-25T09:29:35.941 INFO:teuthology.orchestra.run.smithi153.stderr: 55550K .......... .......... .......... .......... .......... 75% 305M 0s 2019-11-25T09:29:35.941 INFO:teuthology.orchestra.run.smithi153.stderr: 55600K .......... .......... .......... .......... .......... 75% 395M 0s 2019-11-25T09:29:35.941 INFO:teuthology.orchestra.run.smithi153.stderr: 55650K .......... .......... .......... .......... .......... 75% 391M 0s 2019-11-25T09:29:35.941 INFO:teuthology.orchestra.run.smithi153.stderr: 55700K .......... .......... .......... .......... .......... 75% 385M 0s 2019-11-25T09:29:35.942 INFO:teuthology.orchestra.run.smithi153.stderr: 55750K .......... .......... .......... .......... .......... 75% 342M 0s 2019-11-25T09:29:35.942 INFO:teuthology.orchestra.run.smithi153.stderr: 55800K .......... .......... .......... .......... .......... 75% 380M 0s 2019-11-25T09:29:35.942 INFO:teuthology.orchestra.run.smithi153.stderr: 55850K .......... .......... .......... .......... .......... 75% 388M 0s 2019-11-25T09:29:35.942 INFO:teuthology.orchestra.run.smithi153.stderr: 55900K .......... .......... .......... .......... .......... 76% 394M 0s 2019-11-25T09:29:35.942 INFO:teuthology.orchestra.run.smithi153.stderr: 55950K .......... .......... .......... .......... .......... 76% 341M 0s 2019-11-25T09:29:35.942 INFO:teuthology.orchestra.run.smithi153.stderr: 56000K .......... .......... .......... .......... .......... 76% 393M 0s 2019-11-25T09:29:35.942 INFO:teuthology.orchestra.run.smithi153.stderr: 56050K .......... .......... .......... .......... .......... 76% 395M 0s 2019-11-25T09:29:35.943 INFO:teuthology.orchestra.run.smithi153.stderr: 56100K .......... .......... .......... .......... .......... 76% 399M 0s 2019-11-25T09:29:35.943 INFO:teuthology.orchestra.run.smithi153.stderr: 56150K .......... .......... .......... .......... .......... 76% 358M 0s 2019-11-25T09:29:35.943 INFO:teuthology.orchestra.run.smithi153.stderr: 56200K .......... .......... .......... .......... .......... 76% 397M 0s 2019-11-25T09:29:35.943 INFO:teuthology.orchestra.run.smithi153.stderr: 56250K .......... .......... .......... .......... .......... 76% 332M 0s 2019-11-25T09:29:35.943 INFO:teuthology.orchestra.run.smithi153.stderr: 56300K .......... .......... .......... .......... .......... 76% 357M 0s 2019-11-25T09:29:35.943 INFO:teuthology.orchestra.run.smithi153.stderr: 56350K .......... .......... .......... .......... .......... 76% 294M 0s 2019-11-25T09:29:35.943 INFO:teuthology.orchestra.run.smithi153.stderr: 56400K .......... .......... .......... .......... .......... 76% 426M 0s 2019-11-25T09:29:35.944 INFO:teuthology.orchestra.run.smithi153.stderr: 56450K .......... .......... .......... .......... .......... 76% 386M 0s 2019-11-25T09:29:35.944 INFO:teuthology.orchestra.run.smithi153.stderr: 56500K .......... .......... .......... .......... .......... 76% 351M 0s 2019-11-25T09:29:35.944 INFO:teuthology.orchestra.run.smithi153.stderr: 56550K .......... .......... .......... .......... .......... 76% 327M 0s 2019-11-25T09:29:35.944 INFO:teuthology.orchestra.run.smithi153.stderr: 56600K .......... .......... .......... .......... .......... 77% 358M 0s 2019-11-25T09:29:35.944 INFO:teuthology.orchestra.run.smithi153.stderr: 56650K .......... .......... .......... .......... .......... 77% 359M 0s 2019-11-25T09:29:35.944 INFO:teuthology.orchestra.run.smithi153.stderr: 56700K .......... .......... .......... .......... .......... 77% 389M 0s 2019-11-25T09:29:35.944 INFO:teuthology.orchestra.run.smithi153.stderr: 56750K .......... .......... .......... .......... .......... 77% 324M 0s 2019-11-25T09:29:35.945 INFO:teuthology.orchestra.run.smithi153.stderr: 56800K .......... .......... .......... .......... .......... 77% 347M 0s 2019-11-25T09:29:35.945 INFO:teuthology.orchestra.run.smithi153.stderr: 56850K .......... .......... .......... .......... .......... 77% 383M 0s 2019-11-25T09:29:35.945 INFO:teuthology.orchestra.run.smithi153.stderr: 56900K .......... .......... .......... .......... .......... 77% 355M 0s 2019-11-25T09:29:35.945 INFO:teuthology.orchestra.run.smithi153.stderr: 56950K .......... .......... .......... .......... .......... 77% 345M 0s 2019-11-25T09:29:35.945 INFO:teuthology.orchestra.run.smithi153.stderr: 57000K .......... .......... .......... .......... .......... 77% 390M 0s 2019-11-25T09:29:35.945 INFO:teuthology.orchestra.run.smithi153.stderr: 57050K .......... .......... .......... .......... .......... 77% 388M 0s 2019-11-25T09:29:35.945 INFO:teuthology.orchestra.run.smithi153.stderr: 57100K .......... .......... .......... .......... .......... 77% 403M 0s 2019-11-25T09:29:35.945 INFO:teuthology.orchestra.run.smithi153.stderr: 57150K .......... .......... .......... .......... .......... 77% 324M 0s 2019-11-25T09:29:35.946 INFO:teuthology.orchestra.run.smithi153.stderr: 57200K .......... .......... .......... .......... .......... 77% 383M 0s 2019-11-25T09:29:35.946 INFO:teuthology.orchestra.run.smithi153.stderr: 57250K .......... .......... .......... .......... .......... 77% 381M 0s 2019-11-25T09:29:35.946 INFO:teuthology.orchestra.run.smithi153.stderr: 57300K .......... .......... .......... .......... .......... 77% 325M 0s 2019-11-25T09:29:35.946 INFO:teuthology.orchestra.run.smithi153.stderr: 57350K .......... .......... .......... .......... .......... 78% 277M 0s 2019-11-25T09:29:35.946 INFO:teuthology.orchestra.run.smithi153.stderr: 57400K .......... .......... .......... .......... .......... 78% 316M 0s 2019-11-25T09:29:35.946 INFO:teuthology.orchestra.run.smithi153.stderr: 57450K .......... .......... .......... .......... .......... 78% 322M 0s 2019-11-25T09:29:35.946 INFO:teuthology.orchestra.run.smithi153.stderr: 57500K .......... .......... .......... .......... .......... 78% 321M 0s 2019-11-25T09:29:35.947 INFO:teuthology.orchestra.run.smithi153.stderr: 57550K .......... .......... .......... .......... .......... 78% 261M 0s 2019-11-25T09:29:35.947 INFO:teuthology.orchestra.run.smithi153.stderr: 57600K .......... .......... .......... .......... .......... 78% 303M 0s 2019-11-25T09:29:35.947 INFO:teuthology.orchestra.run.smithi153.stderr: 57650K .......... .......... .......... .......... .......... 78% 304M 0s 2019-11-25T09:29:35.947 INFO:teuthology.orchestra.run.smithi153.stderr: 57700K .......... .......... .......... .......... .......... 78% 296M 0s 2019-11-25T09:29:35.947 INFO:teuthology.orchestra.run.smithi153.stderr: 57750K .......... .......... .......... .......... .......... 78% 271M 0s 2019-11-25T09:29:35.947 INFO:teuthology.orchestra.run.smithi153.stderr: 57800K .......... .......... .......... .......... .......... 78% 302M 0s 2019-11-25T09:29:35.947 INFO:teuthology.orchestra.run.smithi153.stderr: 57850K .......... .......... .......... .......... .......... 78% 298M 0s 2019-11-25T09:29:35.948 INFO:teuthology.orchestra.run.smithi153.stderr: 57900K .......... .......... .......... .......... .......... 78% 292M 0s 2019-11-25T09:29:35.948 INFO:teuthology.orchestra.run.smithi153.stderr: 57950K .......... .......... .......... .......... .......... 78% 254M 0s 2019-11-25T09:29:35.948 INFO:teuthology.orchestra.run.smithi153.stderr: 58000K .......... .......... .......... .......... .......... 78% 301M 0s 2019-11-25T09:29:35.948 INFO:teuthology.orchestra.run.smithi153.stderr: 58050K .......... .......... .......... .......... .......... 78% 267M 0s 2019-11-25T09:29:35.948 INFO:teuthology.orchestra.run.smithi153.stderr: 58100K .......... .......... .......... .......... .......... 79% 314M 0s 2019-11-25T09:29:35.948 INFO:teuthology.orchestra.run.smithi153.stderr: 58150K .......... .......... .......... .......... .......... 79% 276M 0s 2019-11-25T09:29:35.948 INFO:teuthology.orchestra.run.smithi153.stderr: 58200K .......... .......... .......... .......... .......... 79% 317M 0s 2019-11-25T09:29:35.949 INFO:teuthology.orchestra.run.smithi153.stderr: 58250K .......... .......... .......... .......... .......... 79% 321M 0s 2019-11-25T09:29:35.949 INFO:teuthology.orchestra.run.smithi153.stderr: 58300K .......... .......... .......... .......... .......... 79% 304M 0s 2019-11-25T09:29:35.949 INFO:teuthology.orchestra.run.smithi153.stderr: 58350K .......... .......... .......... .......... .......... 79% 276M 0s 2019-11-25T09:29:35.949 INFO:teuthology.orchestra.run.smithi153.stderr: 58400K .......... .......... .......... .......... .......... 79% 317M 0s 2019-11-25T09:29:35.949 INFO:teuthology.orchestra.run.smithi153.stderr: 58450K .......... .......... .......... .......... .......... 79% 324M 0s 2019-11-25T09:29:35.949 INFO:teuthology.orchestra.run.smithi153.stderr: 58500K .......... .......... .......... .......... .......... 79% 310M 0s 2019-11-25T09:29:35.949 INFO:teuthology.orchestra.run.smithi153.stderr: 58550K .......... .......... .......... .......... .......... 79% 273M 0s 2019-11-25T09:29:35.950 INFO:teuthology.orchestra.run.smithi153.stderr: 58600K .......... .......... .......... .......... .......... 79% 314M 0s 2019-11-25T09:29:35.950 INFO:teuthology.orchestra.run.smithi153.stderr: 58650K .......... .......... .......... .......... .......... 79% 322M 0s 2019-11-25T09:29:35.950 INFO:teuthology.orchestra.run.smithi153.stderr: 58700K .......... .......... .......... .......... .......... 79% 308M 0s 2019-11-25T09:29:35.950 INFO:teuthology.orchestra.run.smithi153.stderr: 58750K .......... .......... .......... .......... .......... 79% 277M 0s 2019-11-25T09:29:35.950 INFO:teuthology.orchestra.run.smithi153.stderr: 58800K .......... .......... .......... .......... .......... 80% 307M 0s 2019-11-25T09:29:35.950 INFO:teuthology.orchestra.run.smithi153.stderr: 58850K .......... .......... .......... .......... .......... 80% 324M 0s 2019-11-25T09:29:35.950 INFO:teuthology.orchestra.run.smithi153.stderr: 58900K .......... .......... .......... .......... .......... 80% 321M 0s 2019-11-25T09:29:35.950 INFO:teuthology.orchestra.run.smithi153.stderr: 58950K .......... .......... .......... .......... .......... 80% 281M 0s 2019-11-25T09:29:35.951 INFO:teuthology.orchestra.run.smithi153.stderr: 59000K .......... .......... .......... .......... .......... 80% 321M 0s 2019-11-25T09:29:35.951 INFO:teuthology.orchestra.run.smithi153.stderr: 59050K .......... .......... .......... .......... .......... 80% 276M 0s 2019-11-25T09:29:35.951 INFO:teuthology.orchestra.run.smithi153.stderr: 59100K .......... .......... .......... .......... .......... 80% 309M 0s 2019-11-25T09:29:35.951 INFO:teuthology.orchestra.run.smithi153.stderr: 59150K .......... .......... .......... .......... .......... 80% 265M 0s 2019-11-25T09:29:35.951 INFO:teuthology.orchestra.run.smithi153.stderr: 59200K .......... .......... .......... .......... .......... 80% 319M 0s 2019-11-25T09:29:35.951 INFO:teuthology.orchestra.run.smithi153.stderr: 59250K .......... .......... .......... .......... .......... 80% 315M 0s 2019-11-25T09:29:35.951 INFO:teuthology.orchestra.run.smithi153.stderr: 59300K .......... .......... .......... .......... .......... 80% 319M 0s 2019-11-25T09:29:35.952 INFO:teuthology.orchestra.run.smithi153.stderr: 59350K .......... .......... .......... .......... .......... 80% 292M 0s 2019-11-25T09:29:35.952 INFO:teuthology.orchestra.run.smithi153.stderr: 59400K .......... .......... .......... .......... .......... 80% 305M 0s 2019-11-25T09:29:35.952 INFO:teuthology.orchestra.run.smithi153.stderr: 59450K .......... .......... .......... .......... .......... 80% 316M 0s 2019-11-25T09:29:35.952 INFO:teuthology.orchestra.run.smithi153.stderr: 59500K .......... .......... .......... .......... .......... 80% 325M 0s 2019-11-25T09:29:35.952 INFO:teuthology.orchestra.run.smithi153.stderr: 59550K .......... .......... .......... .......... .......... 81% 273M 0s 2019-11-25T09:29:35.952 INFO:teuthology.orchestra.run.smithi153.stderr: 59600K .......... .......... .......... .......... .......... 81% 330M 0s 2019-11-25T09:29:35.952 INFO:teuthology.orchestra.run.smithi153.stderr: 59650K .......... .......... .......... .......... .......... 81% 321M 0s 2019-11-25T09:29:35.953 INFO:teuthology.orchestra.run.smithi153.stderr: 59700K .......... .......... .......... .......... .......... 81% 322M 0s 2019-11-25T09:29:35.953 INFO:teuthology.orchestra.run.smithi153.stderr: 59750K .......... .......... .......... .......... .......... 81% 281M 0s 2019-11-25T09:29:35.953 INFO:teuthology.orchestra.run.smithi153.stderr: 59800K .......... .......... .......... .......... .......... 81% 315M 0s 2019-11-25T09:29:35.953 INFO:teuthology.orchestra.run.smithi153.stderr: 59850K .......... .......... .......... .......... .......... 81% 323M 0s 2019-11-25T09:29:35.953 INFO:teuthology.orchestra.run.smithi153.stderr: 59900K .......... .......... .......... .......... .......... 81% 317M 0s 2019-11-25T09:29:35.953 INFO:teuthology.orchestra.run.smithi153.stderr: 59950K .......... .......... .......... .......... .......... 81% 273M 0s 2019-11-25T09:29:35.953 INFO:teuthology.orchestra.run.smithi153.stderr: 60000K .......... .......... .......... .......... .......... 81% 315M 0s 2019-11-25T09:29:35.954 INFO:teuthology.orchestra.run.smithi153.stderr: 60050K .......... .......... .......... .......... .......... 81% 317M 0s 2019-11-25T09:29:35.954 INFO:teuthology.orchestra.run.smithi153.stderr: 60100K .......... .......... .......... .......... .......... 81% 329M 0s 2019-11-25T09:29:35.954 INFO:teuthology.orchestra.run.smithi153.stderr: 60150K .......... .......... .......... .......... .......... 81% 290M 0s 2019-11-25T09:29:35.954 INFO:teuthology.orchestra.run.smithi153.stderr: 60200K .......... .......... .......... .......... .......... 81% 326M 0s 2019-11-25T09:29:35.954 INFO:teuthology.orchestra.run.smithi153.stderr: 60250K .......... .......... .......... .......... .......... 81% 313M 0s 2019-11-25T09:29:35.954 INFO:teuthology.orchestra.run.smithi153.stderr: 60300K .......... .......... .......... .......... .......... 82% 317M 0s 2019-11-25T09:29:35.954 INFO:teuthology.orchestra.run.smithi153.stderr: 60350K .......... .......... .......... .......... .......... 82% 249M 0s 2019-11-25T09:29:35.955 INFO:teuthology.orchestra.run.smithi153.stderr: 60400K .......... .......... .......... .......... .......... 82% 316M 0s 2019-11-25T09:29:35.955 INFO:teuthology.orchestra.run.smithi153.stderr: 60450K .......... .......... .......... .......... .......... 82% 315M 0s 2019-11-25T09:29:35.955 INFO:teuthology.orchestra.run.smithi153.stderr: 60500K .......... .......... .......... .......... .......... 82% 321M 0s 2019-11-25T09:29:35.955 INFO:teuthology.orchestra.run.smithi153.stderr: 60550K .......... .......... .......... .......... .......... 82% 250M 0s 2019-11-25T09:29:35.955 INFO:teuthology.orchestra.run.smithi153.stderr: 60600K .......... .......... .......... .......... .......... 82% 318M 0s 2019-11-25T09:29:35.955 INFO:teuthology.orchestra.run.smithi153.stderr: 60650K .......... .......... .......... .......... .......... 82% 319M 0s 2019-11-25T09:29:35.955 INFO:teuthology.orchestra.run.smithi153.stderr: 60700K .......... .......... .......... .......... .......... 82% 307M 0s 2019-11-25T09:29:35.956 INFO:teuthology.orchestra.run.smithi153.stderr: 60750K .......... .......... .......... .......... .......... 82% 238M 0s 2019-11-25T09:29:35.956 INFO:teuthology.orchestra.run.smithi153.stderr: 60800K .......... .......... .......... .......... .......... 82% 276M 0s 2019-11-25T09:29:35.956 INFO:teuthology.orchestra.run.smithi153.stderr: 60850K .......... .......... .......... .......... .......... 82% 308M 0s 2019-11-25T09:29:35.956 INFO:teuthology.orchestra.run.smithi153.stderr: 60900K .......... .......... .......... .......... .......... 82% 291M 0s 2019-11-25T09:29:35.956 INFO:teuthology.orchestra.run.smithi153.stderr: 60950K .......... .......... .......... .......... .......... 82% 243M 0s 2019-11-25T09:29:35.956 INFO:teuthology.orchestra.run.smithi153.stderr: 61000K .......... .......... .......... .......... .......... 82% 302M 0s 2019-11-25T09:29:35.956 INFO:teuthology.orchestra.run.smithi153.stderr: 61050K .......... .......... .......... .......... .......... 83% 293M 0s 2019-11-25T09:29:35.957 INFO:teuthology.orchestra.run.smithi153.stderr: 61100K .......... .......... .......... .......... .......... 83% 295M 0s 2019-11-25T09:29:35.957 INFO:teuthology.orchestra.run.smithi153.stderr: 61150K .......... .......... .......... .......... .......... 83% 275M 0s 2019-11-25T09:29:35.957 INFO:teuthology.orchestra.run.smithi153.stderr: 61200K .......... .......... .......... .......... .......... 83% 304M 0s 2019-11-25T09:29:35.957 INFO:teuthology.orchestra.run.smithi153.stderr: 61250K .......... .......... .......... .......... .......... 83% 307M 0s 2019-11-25T09:29:35.957 INFO:teuthology.orchestra.run.smithi153.stderr: 61300K .......... .......... .......... .......... .......... 83% 306M 0s 2019-11-25T09:29:35.957 INFO:teuthology.orchestra.run.smithi153.stderr: 61350K .......... .......... .......... .......... .......... 83% 291M 0s 2019-11-25T09:29:35.957 INFO:teuthology.orchestra.run.smithi153.stderr: 61400K .......... .......... .......... .......... .......... 83% 319M 0s 2019-11-25T09:29:35.958 INFO:teuthology.orchestra.run.smithi153.stderr: 61450K .......... .......... .......... .......... .......... 83% 339M 0s 2019-11-25T09:29:35.958 INFO:teuthology.orchestra.run.smithi153.stderr: 61500K .......... .......... .......... .......... .......... 83% 323M 0s 2019-11-25T09:29:35.958 INFO:teuthology.orchestra.run.smithi153.stderr: 61550K .......... .......... .......... .......... .......... 83% 268M 0s 2019-11-25T09:29:35.958 INFO:teuthology.orchestra.run.smithi153.stderr: 61600K .......... .......... .......... .......... .......... 83% 319M 0s 2019-11-25T09:29:35.958 INFO:teuthology.orchestra.run.smithi153.stderr: 61650K .......... .......... .......... .......... .......... 83% 323M 0s 2019-11-25T09:29:35.958 INFO:teuthology.orchestra.run.smithi153.stderr: 61700K .......... .......... .......... .......... .......... 83% 320M 0s 2019-11-25T09:29:35.958 INFO:teuthology.orchestra.run.smithi153.stderr: 61750K .......... .......... .......... .......... .......... 84% 264M 0s 2019-11-25T09:29:35.959 INFO:teuthology.orchestra.run.smithi153.stderr: 61800K .......... .......... .......... .......... .......... 84% 327M 0s 2019-11-25T09:29:35.959 INFO:teuthology.orchestra.run.smithi153.stderr: 61850K .......... .......... .......... .......... .......... 84% 321M 0s 2019-11-25T09:29:35.959 INFO:teuthology.orchestra.run.smithi153.stderr: 61900K .......... .......... .......... .......... .......... 84% 301M 0s 2019-11-25T09:29:35.959 INFO:teuthology.orchestra.run.smithi153.stderr: 61950K .......... .......... .......... .......... .......... 84% 283M 0s 2019-11-25T09:29:35.959 INFO:teuthology.orchestra.run.smithi153.stderr: 62000K .......... .......... .......... .......... .......... 84% 317M 0s 2019-11-25T09:29:35.959 INFO:teuthology.orchestra.run.smithi153.stderr: 62050K .......... .......... .......... .......... .......... 84% 325M 0s 2019-11-25T09:29:35.959 INFO:teuthology.orchestra.run.smithi153.stderr: 62100K .......... .......... .......... .......... .......... 84% 312M 0s 2019-11-25T09:29:35.959 INFO:teuthology.orchestra.run.smithi153.stderr: 62150K .......... .......... .......... .......... .......... 84% 271M 0s 2019-11-25T09:29:35.960 INFO:teuthology.orchestra.run.smithi153.stderr: 62200K .......... .......... .......... .......... .......... 84% 308M 0s 2019-11-25T09:29:35.960 INFO:teuthology.orchestra.run.smithi153.stderr: 62250K .......... .......... .......... .......... .......... 84% 317M 0s 2019-11-25T09:29:35.960 INFO:teuthology.orchestra.run.smithi153.stderr: 62300K .......... .......... .......... .......... .......... 84% 326M 0s 2019-11-25T09:29:35.960 INFO:teuthology.orchestra.run.smithi153.stderr: 62350K .......... .......... .......... .......... .......... 84% 288M 0s 2019-11-25T09:29:35.960 INFO:teuthology.orchestra.run.smithi153.stderr: 62400K .......... .......... .......... .......... .......... 84% 318M 0s 2019-11-25T09:29:35.960 INFO:teuthology.orchestra.run.smithi153.stderr: 62450K .......... .......... .......... .......... .......... 84% 325M 0s 2019-11-25T09:29:35.960 INFO:teuthology.orchestra.run.smithi153.stderr: 62500K .......... .......... .......... .......... .......... 85% 318M 0s 2019-11-25T09:29:35.961 INFO:teuthology.orchestra.run.smithi153.stderr: 62550K .......... .......... .......... .......... .......... 85% 251M 0s 2019-11-25T09:29:35.961 INFO:teuthology.orchestra.run.smithi153.stderr: 62600K .......... .......... .......... .......... .......... 85% 312M 0s 2019-11-25T09:29:35.961 INFO:teuthology.orchestra.run.smithi153.stderr: 62650K .......... .......... .......... .......... .......... 85% 294M 0s 2019-11-25T09:29:35.961 INFO:teuthology.orchestra.run.smithi153.stderr: 62700K .......... .......... .......... .......... .......... 85% 306M 0s 2019-11-25T09:29:35.961 INFO:teuthology.orchestra.run.smithi153.stderr: 62750K .......... .......... .......... .......... .......... 85% 252M 0s 2019-11-25T09:29:35.961 INFO:teuthology.orchestra.run.smithi153.stderr: 62800K .......... .......... .......... .......... .......... 85% 308M 0s 2019-11-25T09:29:35.961 INFO:teuthology.orchestra.run.smithi153.stderr: 62850K .......... .......... .......... .......... .......... 85% 287M 0s 2019-11-25T09:29:35.962 INFO:teuthology.orchestra.run.smithi153.stderr: 62900K .......... .......... .......... .......... .......... 85% 311M 0s 2019-11-25T09:29:35.962 INFO:teuthology.orchestra.run.smithi153.stderr: 62950K .......... .......... .......... .......... .......... 85% 237M 0s 2019-11-25T09:29:35.962 INFO:teuthology.orchestra.run.smithi153.stderr: 63000K .......... .......... .......... .......... .......... 85% 255M 0s 2019-11-25T09:29:35.962 INFO:teuthology.orchestra.run.smithi153.stderr: 63050K .......... .......... .......... .......... .......... 85% 269M 0s 2019-11-25T09:29:35.962 INFO:teuthology.orchestra.run.smithi153.stderr: 63100K .......... .......... .......... .......... .......... 85% 272M 0s 2019-11-25T09:29:35.962 INFO:teuthology.orchestra.run.smithi153.stderr: 63150K .......... .......... .......... .......... .......... 85% 236M 0s 2019-11-25T09:29:35.962 INFO:teuthology.orchestra.run.smithi153.stderr: 63200K .......... .......... .......... .......... .......... 85% 270M 0s 2019-11-25T09:29:35.963 INFO:teuthology.orchestra.run.smithi153.stderr: 63250K .......... .......... .......... .......... .......... 86% 282M 0s 2019-11-25T09:29:35.963 INFO:teuthology.orchestra.run.smithi153.stderr: 63300K .......... .......... .......... .......... .......... 86% 288M 0s 2019-11-25T09:29:35.963 INFO:teuthology.orchestra.run.smithi153.stderr: 63350K .......... .......... .......... .......... .......... 86% 242M 0s 2019-11-25T09:29:35.963 INFO:teuthology.orchestra.run.smithi153.stderr: 63400K .......... .......... .......... .......... .......... 86% 272M 0s 2019-11-25T09:29:35.963 INFO:teuthology.orchestra.run.smithi153.stderr: 63450K .......... .......... .......... .......... .......... 86% 289M 0s 2019-11-25T09:29:35.963 INFO:teuthology.orchestra.run.smithi153.stderr: 63500K .......... .......... .......... .......... .......... 86% 305M 0s 2019-11-25T09:29:35.963 INFO:teuthology.orchestra.run.smithi153.stderr: 63550K .......... .......... .......... .......... .......... 86% 273M 0s 2019-11-25T09:29:35.964 INFO:teuthology.orchestra.run.smithi153.stderr: 63600K .......... .......... .......... .......... .......... 86% 298M 0s 2019-11-25T09:29:35.964 INFO:teuthology.orchestra.run.smithi153.stderr: 63650K .......... .......... .......... .......... .......... 86% 291M 0s 2019-11-25T09:29:35.964 INFO:teuthology.orchestra.run.smithi153.stderr: 63700K .......... .......... .......... .......... .......... 86% 280M 0s 2019-11-25T09:29:35.964 INFO:teuthology.orchestra.run.smithi153.stderr: 63750K .......... .......... .......... .......... .......... 86% 242M 0s 2019-11-25T09:29:35.964 INFO:teuthology.orchestra.run.smithi153.stderr: 63800K .......... .......... .......... .......... .......... 86% 287M 0s 2019-11-25T09:29:35.964 INFO:teuthology.orchestra.run.smithi153.stderr: 63850K .......... .......... .......... .......... .......... 86% 299M 0s 2019-11-25T09:29:35.964 INFO:teuthology.orchestra.run.smithi153.stderr: 63900K .......... .......... .......... .......... .......... 86% 302M 0s 2019-11-25T09:29:35.965 INFO:teuthology.orchestra.run.smithi153.stderr: 63950K .......... .......... .......... .......... .......... 87% 260M 0s 2019-11-25T09:29:35.965 INFO:teuthology.orchestra.run.smithi153.stderr: 64000K .......... .......... .......... .......... .......... 87% 278M 0s 2019-11-25T09:29:35.965 INFO:teuthology.orchestra.run.smithi153.stderr: 64050K .......... .......... .......... .......... .......... 87% 284M 0s 2019-11-25T09:29:35.965 INFO:teuthology.orchestra.run.smithi153.stderr: 64100K .......... .......... .......... .......... .......... 87% 309M 0s 2019-11-25T09:29:35.965 INFO:teuthology.orchestra.run.smithi153.stderr: 64150K .......... .......... .......... .......... .......... 87% 253M 0s 2019-11-25T09:29:35.965 INFO:teuthology.orchestra.run.smithi153.stderr: 64200K .......... .......... .......... .......... .......... 87% 272M 0s 2019-11-25T09:29:35.965 INFO:teuthology.orchestra.run.smithi153.stderr: 64250K .......... .......... .......... .......... .......... 87% 289M 0s 2019-11-25T09:29:35.966 INFO:teuthology.orchestra.run.smithi153.stderr: 64300K .......... .......... .......... .......... .......... 87% 289M 0s 2019-11-25T09:29:35.966 INFO:teuthology.orchestra.run.smithi153.stderr: 64350K .......... .......... .......... .......... .......... 87% 237M 0s 2019-11-25T09:29:35.966 INFO:teuthology.orchestra.run.smithi153.stderr: 64400K .......... .......... .......... .......... .......... 87% 304M 0s 2019-11-25T09:29:35.966 INFO:teuthology.orchestra.run.smithi153.stderr: 64450K .......... .......... .......... .......... .......... 87% 284M 0s 2019-11-25T09:29:35.966 INFO:teuthology.orchestra.run.smithi153.stderr: 64500K .......... .......... .......... .......... .......... 87% 282M 0s 2019-11-25T09:29:35.966 INFO:teuthology.orchestra.run.smithi153.stderr: 64550K .......... .......... .......... .......... .......... 87% 242M 0s 2019-11-25T09:29:35.966 INFO:teuthology.orchestra.run.smithi153.stderr: 64600K .......... .......... .......... .......... .......... 87% 284M 0s 2019-11-25T09:29:35.966 INFO:teuthology.orchestra.run.smithi153.stderr: 64650K .......... .......... .......... .......... .......... 87% 294M 0s 2019-11-25T09:29:35.967 INFO:teuthology.orchestra.run.smithi153.stderr: 64700K .......... .......... .......... .......... .......... 88% 278M 0s 2019-11-25T09:29:35.967 INFO:teuthology.orchestra.run.smithi153.stderr: 64750K .......... .......... .......... .......... .......... 88% 237M 0s 2019-11-25T09:29:35.967 INFO:teuthology.orchestra.run.smithi153.stderr: 64800K .......... .......... .......... .......... .......... 88% 296M 0s 2019-11-25T09:29:35.967 INFO:teuthology.orchestra.run.smithi153.stderr: 64850K .......... .......... .......... .......... .......... 88% 272M 0s 2019-11-25T09:29:35.967 INFO:teuthology.orchestra.run.smithi153.stderr: 64900K .......... .......... .......... .......... .......... 88% 299M 0s 2019-11-25T09:29:35.967 INFO:teuthology.orchestra.run.smithi153.stderr: 64950K .......... .......... .......... .......... .......... 88% 254M 0s 2019-11-25T09:29:35.967 INFO:teuthology.orchestra.run.smithi153.stderr: 65000K .......... .......... .......... .......... .......... 88% 299M 0s 2019-11-25T09:29:35.968 INFO:teuthology.orchestra.run.smithi153.stderr: 65050K .......... .......... .......... .......... .......... 88% 288M 0s 2019-11-25T09:29:35.968 INFO:teuthology.orchestra.run.smithi153.stderr: 65100K .......... .......... .......... .......... .......... 88% 304M 0s 2019-11-25T09:29:35.968 INFO:teuthology.orchestra.run.smithi153.stderr: 65150K .......... .......... .......... .......... .......... 88% 279M 0s 2019-11-25T09:29:35.968 INFO:teuthology.orchestra.run.smithi153.stderr: 65200K .......... .......... .......... .......... .......... 88% 296M 0s 2019-11-25T09:29:35.968 INFO:teuthology.orchestra.run.smithi153.stderr: 65250K .......... .......... .......... .......... .......... 88% 303M 0s 2019-11-25T09:29:35.968 INFO:teuthology.orchestra.run.smithi153.stderr: 65300K .......... .......... .......... .......... .......... 88% 293M 0s 2019-11-25T09:29:35.968 INFO:teuthology.orchestra.run.smithi153.stderr: 65350K .......... .......... .......... .......... .......... 88% 241M 0s 2019-11-25T09:29:35.969 INFO:teuthology.orchestra.run.smithi153.stderr: 65400K .......... .......... .......... .......... .......... 88% 301M 0s 2019-11-25T09:29:35.969 INFO:teuthology.orchestra.run.smithi153.stderr: 65450K .......... .......... .......... .......... .......... 89% 289M 0s 2019-11-25T09:29:35.969 INFO:teuthology.orchestra.run.smithi153.stderr: 65500K .......... .......... .......... .......... .......... 89% 301M 0s 2019-11-25T09:29:35.969 INFO:teuthology.orchestra.run.smithi153.stderr: 65550K .......... .......... .......... .......... .......... 89% 272M 0s 2019-11-25T09:29:35.969 INFO:teuthology.orchestra.run.smithi153.stderr: 65600K .......... .......... .......... .......... .......... 89% 293M 0s 2019-11-25T09:29:35.969 INFO:teuthology.orchestra.run.smithi153.stderr: 65650K .......... .......... .......... .......... .......... 89% 280M 0s 2019-11-25T09:29:35.969 INFO:teuthology.orchestra.run.smithi153.stderr: 65700K .......... .......... .......... .......... .......... 89% 288M 0s 2019-11-25T09:29:35.970 INFO:teuthology.orchestra.run.smithi153.stderr: 65750K .......... .......... .......... .......... .......... 89% 224M 0s 2019-11-25T09:29:35.970 INFO:teuthology.orchestra.run.smithi153.stderr: 65800K .......... .......... .......... .......... .......... 89% 278M 0s 2019-11-25T09:29:35.970 INFO:teuthology.orchestra.run.smithi153.stderr: 65850K .......... .......... .......... .......... .......... 89% 270M 0s 2019-11-25T09:29:35.970 INFO:teuthology.orchestra.run.smithi153.stderr: 65900K .......... .......... .......... .......... .......... 89% 275M 0s 2019-11-25T09:29:35.970 INFO:teuthology.orchestra.run.smithi153.stderr: 65950K .......... .......... .......... .......... .......... 89% 254M 0s 2019-11-25T09:29:35.970 INFO:teuthology.orchestra.run.smithi153.stderr: 66000K .......... .......... .......... .......... .......... 89% 290M 0s 2019-11-25T09:29:35.970 INFO:teuthology.orchestra.run.smithi153.stderr: 66050K .......... .......... .......... .......... .......... 89% 296M 0s 2019-11-25T09:29:35.971 INFO:teuthology.orchestra.run.smithi153.stderr: 66100K .......... .......... .......... .......... .......... 89% 290M 0s 2019-11-25T09:29:35.971 INFO:teuthology.orchestra.run.smithi153.stderr: 66150K .......... .......... .......... .......... .......... 89% 243M 0s 2019-11-25T09:29:35.971 INFO:teuthology.orchestra.run.smithi153.stderr: 66200K .......... .......... .......... .......... .......... 90% 279M 0s 2019-11-25T09:29:35.971 INFO:teuthology.orchestra.run.smithi153.stderr: 66250K .......... .......... .......... .......... .......... 90% 291M 0s 2019-11-25T09:29:35.971 INFO:teuthology.orchestra.run.smithi153.stderr: 66300K .......... .......... .......... .......... .......... 90% 275M 0s 2019-11-25T09:29:35.971 INFO:teuthology.orchestra.run.smithi153.stderr: 66350K .......... .......... .......... .......... .......... 90% 255M 0s 2019-11-25T09:29:35.971 INFO:teuthology.orchestra.run.smithi153.stderr: 66400K .......... .......... .......... .......... .......... 90% 302M 0s 2019-11-25T09:29:35.972 INFO:teuthology.orchestra.run.smithi153.stderr: 66450K .......... .......... .......... .......... .......... 90% 290M 0s 2019-11-25T09:29:35.972 INFO:teuthology.orchestra.run.smithi153.stderr: 66500K .......... .......... .......... .......... .......... 90% 292M 0s 2019-11-25T09:29:35.972 INFO:teuthology.orchestra.run.smithi153.stderr: 66550K .......... .......... .......... .......... .......... 90% 249M 0s 2019-11-25T09:29:35.972 INFO:teuthology.orchestra.run.smithi153.stderr: 66600K .......... .......... .......... .......... .......... 90% 296M 0s 2019-11-25T09:29:35.972 INFO:teuthology.orchestra.run.smithi153.stderr: 66650K .......... .......... .......... .......... .......... 90% 301M 0s 2019-11-25T09:29:35.972 INFO:teuthology.orchestra.run.smithi153.stderr: 66700K .......... .......... .......... .......... .......... 90% 264M 0s 2019-11-25T09:29:35.972 INFO:teuthology.orchestra.run.smithi153.stderr: 66750K .......... .......... .......... .......... .......... 90% 257M 0s 2019-11-25T09:29:35.973 INFO:teuthology.orchestra.run.smithi153.stderr: 66800K .......... .......... .......... .......... .......... 90% 283M 0s 2019-11-25T09:29:35.973 INFO:teuthology.orchestra.run.smithi153.stderr: 66850K .......... .......... .......... .......... .......... 90% 280M 0s 2019-11-25T09:29:35.973 INFO:teuthology.orchestra.run.smithi153.stderr: 66900K .......... .......... .......... .......... .......... 91% 283M 0s 2019-11-25T09:29:35.973 INFO:teuthology.orchestra.run.smithi153.stderr: 66950K .......... .......... .......... .......... .......... 91% 235M 0s 2019-11-25T09:29:35.973 INFO:teuthology.orchestra.run.smithi153.stderr: 67000K .......... .......... .......... .......... .......... 91% 261M 0s 2019-11-25T09:29:35.973 INFO:teuthology.orchestra.run.smithi153.stderr: 67050K .......... .......... .......... .......... .......... 91% 289M 0s 2019-11-25T09:29:35.973 INFO:teuthology.orchestra.run.smithi153.stderr: 67100K .......... .......... .......... .......... .......... 91% 339M 0s 2019-11-25T09:29:35.974 INFO:teuthology.orchestra.run.smithi153.stderr: 67150K .......... .......... .......... .......... .......... 91% 330M 0s 2019-11-25T09:29:35.974 INFO:teuthology.orchestra.run.smithi153.stderr: 67200K .......... .......... .......... .......... .......... 91% 360M 0s 2019-11-25T09:29:35.974 INFO:teuthology.orchestra.run.smithi153.stderr: 67250K .......... .......... .......... .......... .......... 91% 383M 0s 2019-11-25T09:29:35.974 INFO:teuthology.orchestra.run.smithi153.stderr: 67300K .......... .......... .......... .......... .......... 91% 366M 0s 2019-11-25T09:29:35.974 INFO:teuthology.orchestra.run.smithi153.stderr: 67350K .......... .......... .......... .......... .......... 91% 311M 0s 2019-11-25T09:29:35.974 INFO:teuthology.orchestra.run.smithi153.stderr: 67400K .......... .......... .......... .......... .......... 91% 375M 0s 2019-11-25T09:29:35.974 INFO:teuthology.orchestra.run.smithi153.stderr: 67450K .......... .......... .......... .......... .......... 91% 392M 0s 2019-11-25T09:29:35.975 INFO:teuthology.orchestra.run.smithi153.stderr: 67500K .......... .......... .......... .......... .......... 91% 386M 0s 2019-11-25T09:29:35.975 INFO:teuthology.orchestra.run.smithi153.stderr: 67550K .......... .......... .......... .......... .......... 91% 325M 0s 2019-11-25T09:29:35.975 INFO:teuthology.orchestra.run.smithi153.stderr: 67600K .......... .......... .......... .......... .......... 91% 378M 0s 2019-11-25T09:29:35.975 INFO:teuthology.orchestra.run.smithi153.stderr: 67650K .......... .......... .......... .......... .......... 92% 383M 0s 2019-11-25T09:29:35.975 INFO:teuthology.orchestra.run.smithi153.stderr: 67700K .......... .......... .......... .......... .......... 92% 398M 0s 2019-11-25T09:29:35.975 INFO:teuthology.orchestra.run.smithi153.stderr: 67750K .......... .......... .......... .......... .......... 92% 334M 0s 2019-11-25T09:29:35.975 INFO:teuthology.orchestra.run.smithi153.stderr: 67800K .......... .......... .......... .......... .......... 92% 401M 0s 2019-11-25T09:29:35.975 INFO:teuthology.orchestra.run.smithi153.stderr: 67850K .......... .......... .......... .......... .......... 92% 373M 0s 2019-11-25T09:29:35.976 INFO:teuthology.orchestra.run.smithi153.stderr: 67900K .......... .......... .......... .......... .......... 92% 398M 0s 2019-11-25T09:29:35.976 INFO:teuthology.orchestra.run.smithi153.stderr: 67950K .......... .......... .......... .......... .......... 92% 352M 0s 2019-11-25T09:29:35.976 INFO:teuthology.orchestra.run.smithi153.stderr: 68000K .......... .......... .......... .......... .......... 92% 419M 0s 2019-11-25T09:29:35.976 INFO:teuthology.orchestra.run.smithi153.stderr: 68050K .......... .......... .......... .......... .......... 92% 421M 0s 2019-11-25T09:29:35.976 INFO:teuthology.orchestra.run.smithi153.stderr: 68100K .......... .......... .......... .......... .......... 92% 384M 0s 2019-11-25T09:29:35.976 INFO:teuthology.orchestra.run.smithi153.stderr: 68150K .......... .......... .......... .......... .......... 92% 319M 0s 2019-11-25T09:29:35.976 INFO:teuthology.orchestra.run.smithi153.stderr: 68200K .......... .......... .......... .......... .......... 92% 361M 0s 2019-11-25T09:29:35.977 INFO:teuthology.orchestra.run.smithi153.stderr: 68250K .......... .......... .......... .......... .......... 92% 414M 0s 2019-11-25T09:29:35.977 INFO:teuthology.orchestra.run.smithi153.stderr: 68300K .......... .......... .......... .......... .......... 92% 392M 0s 2019-11-25T09:29:35.977 INFO:teuthology.orchestra.run.smithi153.stderr: 68350K .......... .......... .......... .......... .......... 92% 366M 0s 2019-11-25T09:29:35.977 INFO:teuthology.orchestra.run.smithi153.stderr: 68400K .......... .......... .......... .......... .......... 93% 402M 0s 2019-11-25T09:29:35.977 INFO:teuthology.orchestra.run.smithi153.stderr: 68450K .......... .......... .......... .......... .......... 93% 378M 0s 2019-11-25T09:29:35.977 INFO:teuthology.orchestra.run.smithi153.stderr: 68500K .......... .......... .......... .......... .......... 93% 391M 0s 2019-11-25T09:29:35.977 INFO:teuthology.orchestra.run.smithi153.stderr: 68550K .......... .......... .......... .......... .......... 93% 284M 0s 2019-11-25T09:29:35.978 INFO:teuthology.orchestra.run.smithi153.stderr: 68600K .......... .......... .......... .......... .......... 93% 328M 0s 2019-11-25T09:29:35.978 INFO:teuthology.orchestra.run.smithi153.stderr: 68650K .......... .......... .......... .......... .......... 93% 342M 0s 2019-11-25T09:29:35.978 INFO:teuthology.orchestra.run.smithi153.stderr: 68700K .......... .......... .......... .......... .......... 93% 315M 0s 2019-11-25T09:29:35.978 INFO:teuthology.orchestra.run.smithi153.stderr: 68750K .......... .......... .......... .......... .......... 93% 306M 0s 2019-11-25T09:29:35.978 INFO:teuthology.orchestra.run.smithi153.stderr: 68800K .......... .......... .......... .......... .......... 93% 379M 0s 2019-11-25T09:29:35.978 INFO:teuthology.orchestra.run.smithi153.stderr: 68850K .......... .......... .......... .......... .......... 93% 387M 0s 2019-11-25T09:29:35.978 INFO:teuthology.orchestra.run.smithi153.stderr: 68900K .......... .......... .......... .......... .......... 93% 378M 0s 2019-11-25T09:29:35.979 INFO:teuthology.orchestra.run.smithi153.stderr: 68950K .......... .......... .......... .......... .......... 93% 292M 0s 2019-11-25T09:29:35.979 INFO:teuthology.orchestra.run.smithi153.stderr: 69000K .......... .......... .......... .......... .......... 93% 342M 0s 2019-11-25T09:29:35.979 INFO:teuthology.orchestra.run.smithi153.stderr: 69050K .......... .......... .......... .......... .......... 93% 397M 0s 2019-11-25T09:29:35.979 INFO:teuthology.orchestra.run.smithi153.stderr: 69100K .......... .......... .......... .......... .......... 94% 385M 0s 2019-11-25T09:29:35.979 INFO:teuthology.orchestra.run.smithi153.stderr: 69150K .......... .......... .......... .......... .......... 94% 337M 0s 2019-11-25T09:29:35.979 INFO:teuthology.orchestra.run.smithi153.stderr: 69200K .......... .......... .......... .......... .......... 94% 381M 0s 2019-11-25T09:29:35.979 INFO:teuthology.orchestra.run.smithi153.stderr: 69250K .......... .......... .......... .......... .......... 94% 374M 0s 2019-11-25T09:29:35.980 INFO:teuthology.orchestra.run.smithi153.stderr: 69300K .......... .......... .......... .......... .......... 94% 408M 0s 2019-11-25T09:29:35.980 INFO:teuthology.orchestra.run.smithi153.stderr: 69350K .......... .......... .......... .......... .......... 94% 347M 0s 2019-11-25T09:29:35.980 INFO:teuthology.orchestra.run.smithi153.stderr: 69400K .......... .......... .......... .......... .......... 94% 384M 0s 2019-11-25T09:29:35.980 INFO:teuthology.orchestra.run.smithi153.stderr: 69450K .......... .......... .......... .......... .......... 94% 422M 0s 2019-11-25T09:29:35.980 INFO:teuthology.orchestra.run.smithi153.stderr: 69500K .......... .......... .......... .......... .......... 94% 426M 0s 2019-11-25T09:29:35.980 INFO:teuthology.orchestra.run.smithi153.stderr: 69550K .......... .......... .......... .......... .......... 94% 358M 0s 2019-11-25T09:29:35.980 INFO:teuthology.orchestra.run.smithi153.stderr: 69600K .......... .......... .......... .......... .......... 94% 401M 0s 2019-11-25T09:29:35.981 INFO:teuthology.orchestra.run.smithi153.stderr: 69650K .......... .......... .......... .......... .......... 94% 366M 0s 2019-11-25T09:29:35.981 INFO:teuthology.orchestra.run.smithi153.stderr: 69700K .......... .......... .......... .......... .......... 94% 416M 0s 2019-11-25T09:29:35.981 INFO:teuthology.orchestra.run.smithi153.stderr: 69750K .......... .......... .......... .......... .......... 94% 352M 0s 2019-11-25T09:29:35.981 INFO:teuthology.orchestra.run.smithi153.stderr: 69800K .......... .......... .......... .......... .......... 94% 392M 0s 2019-11-25T09:29:35.981 INFO:teuthology.orchestra.run.smithi153.stderr: 69850K .......... .......... .......... .......... .......... 95% 412M 0s 2019-11-25T09:29:35.981 INFO:teuthology.orchestra.run.smithi153.stderr: 69900K .......... .......... .......... .......... .......... 95% 401M 0s 2019-11-25T09:29:35.981 INFO:teuthology.orchestra.run.smithi153.stderr: 69950K .......... .......... .......... .......... .......... 95% 358M 0s 2019-11-25T09:29:35.982 INFO:teuthology.orchestra.run.smithi153.stderr: 70000K .......... .......... .......... .......... .......... 95% 366M 0s 2019-11-25T09:29:35.982 INFO:teuthology.orchestra.run.smithi153.stderr: 70050K .......... .......... .......... .......... .......... 95% 389M 0s 2019-11-25T09:29:35.982 INFO:teuthology.orchestra.run.smithi153.stderr: 70100K .......... .......... .......... .......... .......... 95% 423M 0s 2019-11-25T09:29:35.982 INFO:teuthology.orchestra.run.smithi153.stderr: 70150K .......... .......... .......... .......... .......... 95% 327M 0s 2019-11-25T09:29:35.982 INFO:teuthology.orchestra.run.smithi153.stderr: 70200K .......... .......... .......... .......... .......... 95% 390M 0s 2019-11-25T09:29:35.982 INFO:teuthology.orchestra.run.smithi153.stderr: 70250K .......... .......... .......... .......... .......... 95% 400M 0s 2019-11-25T09:29:35.982 INFO:teuthology.orchestra.run.smithi153.stderr: 70300K .......... .......... .......... .......... .......... 95% 402M 0s 2019-11-25T09:29:35.982 INFO:teuthology.orchestra.run.smithi153.stderr: 70350K .......... .......... .......... .......... .......... 95% 366M 0s 2019-11-25T09:29:35.983 INFO:teuthology.orchestra.run.smithi153.stderr: 70400K .......... .......... .......... .......... .......... 95% 357M 0s 2019-11-25T09:29:35.983 INFO:teuthology.orchestra.run.smithi153.stderr: 70450K .......... .......... .......... .......... .......... 95% 369M 0s 2019-11-25T09:29:35.983 INFO:teuthology.orchestra.run.smithi153.stderr: 70500K .......... .......... .......... .......... .......... 95% 369M 0s 2019-11-25T09:29:35.983 INFO:teuthology.orchestra.run.smithi153.stderr: 70550K .......... .......... .......... .......... .......... 95% 335M 0s 2019-11-25T09:29:35.983 INFO:teuthology.orchestra.run.smithi153.stderr: 70600K .......... .......... .......... .......... .......... 96% 413M 0s 2019-11-25T09:29:35.983 INFO:teuthology.orchestra.run.smithi153.stderr: 70650K .......... .......... .......... .......... .......... 96% 409M 0s 2019-11-25T09:29:35.983 INFO:teuthology.orchestra.run.smithi153.stderr: 70700K .......... .......... .......... .......... .......... 96% 415M 0s 2019-11-25T09:29:35.984 INFO:teuthology.orchestra.run.smithi153.stderr: 70750K .......... .......... .......... .......... .......... 96% 368M 0s 2019-11-25T09:29:35.984 INFO:teuthology.orchestra.run.smithi153.stderr: 70800K .......... .......... .......... .......... .......... 96% 400M 0s 2019-11-25T09:29:35.984 INFO:teuthology.orchestra.run.smithi153.stderr: 70850K .......... .......... .......... .......... .......... 96% 408M 0s 2019-11-25T09:29:35.984 INFO:teuthology.orchestra.run.smithi153.stderr: 70900K .......... .......... .......... .......... .......... 96% 401M 0s 2019-11-25T09:29:35.984 INFO:teuthology.orchestra.run.smithi153.stderr: 70950K .......... .......... .......... .......... .......... 96% 307M 0s 2019-11-25T09:29:35.984 INFO:teuthology.orchestra.run.smithi153.stderr: 71000K .......... .......... .......... .......... .......... 96% 398M 0s 2019-11-25T09:29:35.984 INFO:teuthology.orchestra.run.smithi153.stderr: 71050K .......... .......... .......... .......... .......... 96% 363M 0s 2019-11-25T09:29:35.985 INFO:teuthology.orchestra.run.smithi153.stderr: 71100K .......... .......... .......... .......... .......... 96% 374M 0s 2019-11-25T09:29:35.985 INFO:teuthology.orchestra.run.smithi153.stderr: 71150K .......... .......... .......... .......... .......... 96% 333M 0s 2019-11-25T09:29:35.985 INFO:teuthology.orchestra.run.smithi153.stderr: 71200K .......... .......... .......... .......... .......... 96% 411M 0s 2019-11-25T09:29:35.985 INFO:teuthology.orchestra.run.smithi153.stderr: 71250K .......... .......... .......... .......... .......... 96% 402M 0s 2019-11-25T09:29:35.985 INFO:teuthology.orchestra.run.smithi153.stderr: 71300K .......... .......... .......... .......... .......... 96% 366M 0s 2019-11-25T09:29:35.985 INFO:teuthology.orchestra.run.smithi153.stderr: 71350K .......... .......... .......... .......... .......... 97% 313M 0s 2019-11-25T09:29:35.985 INFO:teuthology.orchestra.run.smithi153.stderr: 71400K .......... .......... .......... .......... .......... 97% 377M 0s 2019-11-25T09:29:35.986 INFO:teuthology.orchestra.run.smithi153.stderr: 71450K .......... .......... .......... .......... .......... 97% 408M 0s 2019-11-25T09:29:35.986 INFO:teuthology.orchestra.run.smithi153.stderr: 71500K .......... .......... .......... .......... .......... 97% 379M 0s 2019-11-25T09:29:35.986 INFO:teuthology.orchestra.run.smithi153.stderr: 71550K .......... .......... .......... .......... .......... 97% 351M 0s 2019-11-25T09:29:35.986 INFO:teuthology.orchestra.run.smithi153.stderr: 71600K .......... .......... .......... .......... .......... 97% 347M 0s 2019-11-25T09:29:35.986 INFO:teuthology.orchestra.run.smithi153.stderr: 71650K .......... .......... .......... .......... .......... 97% 341M 0s 2019-11-25T09:29:35.986 INFO:teuthology.orchestra.run.smithi153.stderr: 71700K .......... .......... .......... .......... .......... 97% 402M 0s 2019-11-25T09:29:35.986 INFO:teuthology.orchestra.run.smithi153.stderr: 71750K .......... .......... .......... .......... .......... 97% 361M 0s 2019-11-25T09:29:35.987 INFO:teuthology.orchestra.run.smithi153.stderr: 71800K .......... .......... .......... .......... .......... 97% 431M 0s 2019-11-25T09:29:35.987 INFO:teuthology.orchestra.run.smithi153.stderr: 71850K .......... .......... .......... .......... .......... 97% 373M 0s 2019-11-25T09:29:35.987 INFO:teuthology.orchestra.run.smithi153.stderr: 71900K .......... .......... .......... .......... .......... 97% 377M 0s 2019-11-25T09:29:35.987 INFO:teuthology.orchestra.run.smithi153.stderr: 71950K .......... .......... .......... .......... .......... 97% 244M 0s 2019-11-25T09:29:35.987 INFO:teuthology.orchestra.run.smithi153.stderr: 72000K .......... .......... .......... .......... .......... 97% 386M 0s 2019-11-25T09:29:35.987 INFO:teuthology.orchestra.run.smithi153.stderr: 72050K .......... .......... .......... .......... .......... 98% 391M 0s 2019-11-25T09:29:35.987 INFO:teuthology.orchestra.run.smithi153.stderr: 72100K .......... .......... .......... .......... .......... 98% 416M 0s 2019-11-25T09:29:35.988 INFO:teuthology.orchestra.run.smithi153.stderr: 72150K .......... .......... .......... .......... .......... 98% 359M 0s 2019-11-25T09:29:35.988 INFO:teuthology.orchestra.run.smithi153.stderr: 72200K .......... .......... .......... .......... .......... 98% 388M 0s 2019-11-25T09:29:35.988 INFO:teuthology.orchestra.run.smithi153.stderr: 72250K .......... .......... .......... .......... .......... 98% 404M 0s 2019-11-25T09:29:35.988 INFO:teuthology.orchestra.run.smithi153.stderr: 72300K .......... .......... .......... .......... .......... 98% 410M 0s 2019-11-25T09:29:35.988 INFO:teuthology.orchestra.run.smithi153.stderr: 72350K .......... .......... .......... .......... .......... 98% 337M 0s 2019-11-25T09:29:35.988 INFO:teuthology.orchestra.run.smithi153.stderr: 72400K .......... .......... .......... .......... .......... 98% 399M 0s 2019-11-25T09:29:35.988 INFO:teuthology.orchestra.run.smithi153.stderr: 72450K .......... .......... .......... .......... .......... 98% 384M 0s 2019-11-25T09:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr: 72500K .......... .......... .......... .......... .......... 98% 394M 0s 2019-11-25T09:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr: 72550K .......... .......... .......... .......... .......... 98% 351M 0s 2019-11-25T09:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr: 72600K .......... .......... .......... .......... .......... 98% 403M 0s 2019-11-25T09:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr: 72650K .......... .......... .......... .......... .......... 98% 397M 0s 2019-11-25T09:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr: 72700K .......... .......... .......... .......... .......... 98% 414M 0s 2019-11-25T09:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr: 72750K .......... .......... .......... .......... .......... 98% 363M 0s 2019-11-25T09:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr: 72800K .......... .......... .......... .......... .......... 99% 428M 0s 2019-11-25T09:29:35.989 INFO:teuthology.orchestra.run.smithi153.stderr: 72850K .......... .......... .......... .......... .......... 99% 397M 0s 2019-11-25T09:29:35.990 INFO:teuthology.orchestra.run.smithi153.stderr: 72900K .......... .......... .......... .......... .......... 99% 405M 0s 2019-11-25T09:29:35.990 INFO:teuthology.orchestra.run.smithi153.stderr: 72950K .......... .......... .......... .......... .......... 99% 360M 0s 2019-11-25T09:29:35.990 INFO:teuthology.orchestra.run.smithi153.stderr: 73000K .......... .......... .......... .......... .......... 99% 373M 0s 2019-11-25T09:29:35.990 INFO:teuthology.orchestra.run.smithi153.stderr: 73050K .......... .......... .......... .......... .......... 99% 384M 0s 2019-11-25T09:29:35.990 INFO:teuthology.orchestra.run.smithi153.stderr: 73100K .......... .......... .......... .......... .......... 99% 425M 0s 2019-11-25T09:29:35.990 INFO:teuthology.orchestra.run.smithi153.stderr: 73150K .......... .......... .......... .......... .......... 99% 331M 0s 2019-11-25T09:29:35.990 INFO:teuthology.orchestra.run.smithi153.stderr: 73200K .......... .......... .......... .......... .......... 99% 414M 0s 2019-11-25T09:29:35.991 INFO:teuthology.orchestra.run.smithi153.stderr: 73250K .......... .......... .......... .......... .......... 99% 398M 0s 2019-11-25T09:29:35.991 INFO:teuthology.orchestra.run.smithi153.stderr: 73300K .......... .......... .......... .......... .......... 99% 404M 0s 2019-11-25T09:29:35.991 INFO:teuthology.orchestra.run.smithi153.stderr: 73350K .......... .......... .......... .......... .......... 99% 353M 0s 2019-11-25T09:29:35.991 INFO:teuthology.orchestra.run.smithi153.stderr: 73400K .......... .......... .......... .......... .......... 99% 409M 0s 2019-11-25T09:29:35.991 INFO:teuthology.orchestra.run.smithi153.stderr: 73450K .......... .......... .......... .......... .......... 99% 415M 0s 2019-11-25T09:29:35.991 INFO:teuthology.orchestra.run.smithi153.stderr: 73500K .......... .......... .......... .......... .......... 99% 418M 0s 2019-11-25T09:29:35.991 INFO:teuthology.orchestra.run.smithi153.stderr: 73550K ........ 100% 368M=0.3s 2019-11-25T09:29:35.992 INFO:teuthology.orchestra.run.smithi153.stderr: 2019-11-25T09:29:35.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25 09:29:34 (240 MB/s) - ‘gradle-4.7-bin.zip’ saved [75323957/75323957] 2019-11-25T09:29:35.992 INFO:teuthology.orchestra.run.smithi153.stderr: 2019-11-25T09:29:35.992 INFO:teuthology.orchestra.run.smithi153.stdout:Archive: gradle-4.7-bin.zip 2019-11-25T09:29:35.992 INFO:teuthology.orchestra.run.smithi153.stdout: creating: /opt/gradle/gradle-4.7/ 2019-11-25T09:29:35.992 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/getting-started.html 2019-11-25T09:29:35.993 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/LICENSE 2019-11-25T09:29:35.993 INFO:teuthology.orchestra.run.smithi153.stdout: creating: /opt/gradle/gradle-4.7/media/ 2019-11-25T09:29:35.993 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-64x64.png 2019-11-25T09:29:35.993 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-48x48.png 2019-11-25T09:29:35.993 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle.icns 2019-11-25T09:29:35.993 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-24x24.png 2019-11-25T09:29:35.993 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-256x256.png 2019-11-25T09:29:35.994 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-512x512.png 2019-11-25T09:29:35.994 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-16x16.png 2019-11-25T09:29:35.994 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-32x32.png 2019-11-25T09:29:35.994 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/media/gradle-icon-128x128.png 2019-11-25T09:29:35.994 INFO:teuthology.orchestra.run.smithi153.stdout: creating: /opt/gradle/gradle-4.7/init.d/ 2019-11-25T09:29:35.994 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/init.d/readme.txt 2019-11-25T09:29:35.994 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/NOTICE 2019-11-25T09:29:35.995 INFO:teuthology.orchestra.run.smithi153.stdout: creating: /opt/gradle/gradle-4.7/bin/ 2019-11-25T09:29:35.995 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle.bat 2019-11-25T09:29:35.995 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/bin/gradle 2019-11-25T09:29:35.995 INFO:teuthology.orchestra.run.smithi153.stdout: creating: /opt/gradle/gradle-4.7/lib/ 2019-11-25T09:29:35.995 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/annotations-13.0.jar 2019-11-25T09:29:35.995 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-0.16.3.jar 2019-11-25T09:29:35.995 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-builders-0.16.3.jar 2019-11-25T09:29:35.995 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-kotlin-dsl-tooling-models-0.16.3.jar 2019-11-25T09:29:35.996 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-compiler-embeddable-1.2.31.jar 2019-11-25T09:29:35.996 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-reflect-1.2.31.jar 2019-11-25T09:29:35.996 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-sam-with-receiver-compiler-plugin-1.2.31.jar 2019-11-25T09:29:35.996 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-script-runtime-1.2.31.jar 2019-11-25T09:29:35.996 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-1.2.31.jar 2019-11-25T09:29:35.996 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk7-1.2.31.jar 2019-11-25T09:29:35.997 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/kotlin-stdlib-jdk8-1.2.31.jar 2019-11-25T09:29:35.997 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-launcher-4.7.jar 2019-11-25T09:29:35.997 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-runtime-api-info-4.7.jar 2019-11-25T09:29:35.997 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-tooling-api-4.7.jar 2019-11-25T09:29:35.997 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-wrapper-4.7.jar 2019-11-25T09:29:35.997 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-installation-beacon-4.7.jar 2019-11-25T09:29:35.997 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-4.7.jar 2019-11-25T09:29:35.997 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-jvm-services-4.7.jar 2019-11-25T09:29:35.998 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-core-api-4.7.jar 2019-11-25T09:29:35.998 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-cache-4.7.jar 2019-11-25T09:29:35.998 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-logging-4.7.jar 2019-11-25T09:29:35.998 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-process-services-4.7.jar 2019-11-25T09:29:35.998 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-resources-4.7.jar 2019-11-25T09:29:35.998 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-persistent-cache-4.7.jar 2019-11-25T09:29:35.998 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-native-4.7.jar 2019-11-25T09:29:35.999 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-groovy-4.7.jar 2019-11-25T09:29:35.999 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-model-core-4.7.jar 2019-11-25T09:29:35.999 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-groovy-4.7.jar 2019-11-25T09:29:35.999 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-messaging-4.7.jar 2019-11-25T09:29:35.999 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-base-services-4.7.jar 2019-11-25T09:29:35.999 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-build-option-4.7.jar 2019-11-25T09:29:35.999 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-cli-4.7.jar 2019-11-25T09:29:36.000 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/gradle-docs-4.7.jar 2019-11-25T09:29:36.000 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-commons-6.1.1.jar 2019-11-25T09:29:36.000 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-analysis-6.1.1.jar 2019-11-25T09:29:36.000 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-tree-6.1.1.jar 2019-11-25T09:29:36.000 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/kryo-2.20.jar 2019-11-25T09:29:36.000 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/reflectasm-1.07-shaded.jar 2019-11-25T09:29:36.000 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/asm-6.1.1.jar 2019-11-25T09:29:36.001 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-io-2.4.jar 2019-11-25T09:29:36.001 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcl-over-slf4j-1.7.16.jar 2019-11-25T09:29:36.001 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/jul-to-slf4j-1.7.16.jar 2019-11-25T09:29:36.001 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/log4j-over-slf4j-1.7.16.jar 2019-11-25T09:29:36.001 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/slf4j-api-1.7.16.jar 2019-11-25T09:29:36.001 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/guava-jdk5-17.0.jar 2019-11-25T09:29:36.001 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/jsr305-2.0.1.jar 2019-11-25T09:29:36.002 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/fastutil-7.2.1-min.jar 2019-11-25T09:29:36.002 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-lang-2.6.jar 2019-11-25T09:29:36.002 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/jcip-annotations-1.0.jar 2019-11-25T09:29:36.002 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/groovy-all-2.4.12.jar 2019-11-25T09:29:36.002 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-1.9.9.jar 2019-11-25T09:29:36.002 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/javax.inject-1.jar 2019-11-25T09:29:36.002 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-collections-3.2.2.jar 2019-11-25T09:29:36.003 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-0.14.jar 2019-11-25T09:29:36.003 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/commons-compress-1.14.jar 2019-11-25T09:29:36.003 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/xml-apis-1.4.01.jar 2019-11-25T09:29:36.003 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/jansi-1.14.jar 2019-11-25T09:29:36.003 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/ant-launcher-1.9.9.jar 2019-11-25T09:29:36.003 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libcpp-0.14.jar 2019-11-25T09:29:36.003 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-amd64-libstdcpp-0.14.jar 2019-11-25T09:29:36.004 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libcpp-0.14.jar 2019-11-25T09:29:36.004 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-freebsd-i386-libstdcpp-0.14.jar 2019-11-25T09:29:36.004 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-0.14.jar 2019-11-25T09:29:36.004 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-0.14.jar 2019-11-25T09:29:36.004 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-amd64-0.14.jar 2019-11-25T09:29:36.004 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-osx-i386-0.14.jar 2019-11-25T09:29:36.004 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-amd64-0.14.jar 2019-11-25T09:29:36.005 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-windows-i386-0.14.jar 2019-11-25T09:29:36.005 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses5-0.14.jar 2019-11-25T09:29:36.005 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-amd64-ncurses6-0.14.jar 2019-11-25T09:29:36.005 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses5-0.14.jar 2019-11-25T09:29:36.005 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/native-platform-linux-i386-ncurses6-0.14.jar 2019-11-25T09:29:36.005 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/minlog-1.2.jar 2019-11-25T09:29:36.005 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/objenesis-1.2.jar 2019-11-25T09:29:36.006 INFO:teuthology.orchestra.run.smithi153.stdout: creating: /opt/gradle/gradle-4.7/lib/plugins/ 2019-11-25T09:29:36.006 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-announce-4.7.jar 2019-11-25T09:29:36.006 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-antlr-4.7.jar 2019-11-25T09:29:36.006 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-code-quality-4.7.jar 2019-11-25T09:29:36.006 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-osgi-4.7.jar 2019-11-25T09:29:36.006 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-signing-4.7.jar 2019-11-25T09:29:36.006 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-native-4.7.jar 2019-11-25T09:29:36.006 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-native-4.7.jar 2019-11-25T09:29:36.007 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-native-4.7.jar 2019-11-25T09:29:36.007 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-development-4.7.jar 2019-11-25T09:29:36.007 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-maven-4.7.jar 2019-11-25T09:29:36.007 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-play-4.7.jar 2019-11-25T09:29:36.007 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ide-4.7.jar 2019-11-25T09:29:36.007 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-scala-4.7.jar 2019-11-25T09:29:36.007 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-comparison-4.7.jar 2019-11-25T09:29:36.008 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ear-4.7.jar 2019-11-25T09:29:36.008 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-play-4.7.jar 2019-11-25T09:29:36.008 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-javascript-4.7.jar 2019-11-25T09:29:36.008 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-ivy-4.7.jar 2019-11-25T09:29:36.008 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-jacoco-4.7.jar 2019-11-25T09:29:36.008 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-init-4.7.jar 2019-11-25T09:29:36.008 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugins-4.7.jar 2019-11-25T09:29:36.009 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-tooling-api-builders-4.7.jar 2019-11-25T09:29:36.009 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-test-kit-4.7.jar 2019-11-25T09:29:36.009 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-composite-builds-4.7.jar 2019-11-25T09:29:36.009 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-groovy-4.7.jar 2019-11-25T09:29:36.009 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-scala-4.7.jar 2019-11-25T09:29:36.009 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-junit-platform-4.7.jar 2019-11-25T09:29:36.009 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-jvm-4.7.jar 2019-11-25T09:29:36.010 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-java-4.7.jar 2019-11-25T09:29:36.010 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-language-jvm-4.7.jar 2019-11-25T09:29:36.010 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-jvm-4.7.jar 2019-11-25T09:29:36.010 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-native-4.7.jar 2019-11-25T09:29:36.010 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-diagnostics-4.7.jar 2019-11-25T09:29:36.010 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-testing-base-4.7.jar 2019-11-25T09:29:36.010 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-reporting-4.7.jar 2019-11-25T09:29:36.011 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-publish-4.7.jar 2019-11-25T09:29:36.011 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-platform-base-4.7.jar 2019-11-25T09:29:36.011 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-plugin-use-4.7.jar 2019-11-25T09:29:36.011 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-s3-4.7.jar 2019-11-25T09:29:36.011 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-gcs-4.7.jar 2019-11-25T09:29:36.011 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-build-cache-http-4.7.jar 2019-11-25T09:29:36.011 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-dependency-management-4.7.jar 2019-11-25T09:29:36.012 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-http-4.7.jar 2019-11-25T09:29:36.012 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-resources-sftp-4.7.jar 2019-11-25T09:29:36.012 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-workers-4.7.jar 2019-11-25T09:29:36.012 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gradle-version-control-4.7.jar 2019-11-25T09:29:36.012 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-s3-1.11.267.jar 2019-11-25T09:29:36.012 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-kms-1.11.267.jar 2019-11-25T09:29:36.012 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aws-java-sdk-core-1.11.267.jar 2019-11-25T09:29:36.013 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/org.eclipse.jgit-4.5.3.201708160445-r.jar 2019-11-25T09:29:36.013 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-2.4.jar 2019-11-25T09:29:36.013 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-http-shared4-2.4.jar 2019-11-25T09:29:36.013 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-services-storage-v1-rev116-1.23.0.jar 2019-11-25T09:29:36.013 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-api-client-1.23.0.jar 2019-11-25T09:29:36.013 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-oauth-client-1.23.0.jar 2019-11-25T09:29:36.013 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-jackson2-1.23.0.jar 2019-11-25T09:29:36.014 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/google-http-client-1.23.0.jar 2019-11-25T09:29:36.014 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpclient-4.5.5.jar 2019-11-25T09:29:36.014 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-codec-1.10.jar 2019-11-25T09:29:36.014 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-container-default-1.5.5.jar 2019-11-25T09:29:36.014 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xbean-reflect-3.4.jar 2019-11-25T09:29:36.014 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/asm-util-6.1.1.jar 2019-11-25T09:29:36.014 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jatl-0.2.2.jar 2019-11-25T09:29:36.015 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/commons-cli-1.2.jar 2019-11-25T09:29:36.015 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/biz.aQute.bndlib-3.4.0.jar 2019-11-25T09:29:36.015 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ivy-2.2.0.jar 2019-11-25T09:29:36.015 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-compat-3.0.4.jar 2019-11-25T09:29:36.015 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-core-3.0.4.jar 2019-11-25T09:29:36.015 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-common-0.8-20100325.jar 2019-11-25T09:29:36.015 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/pmaven-groovy-0.8-20100325.jar 2019-11-25T09:29:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-file-2.4.jar 2019-11-25T09:29:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-connector-wagon-1.13.1.jar 2019-11-25T09:29:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcpg-jdk15on-1.58.jar 2019-11-25T09:29:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/rhino-1.7R3.jar 2019-11-25T09:29:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/gson-2.7.jar 2019-11-25T09:29:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/simple-4.1.21.jar 2019-11-25T09:29:36.016 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-plugin-api-3.0.4.jar 2019-11-25T09:29:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/testng-6.3.1.jar 2019-11-25T09:29:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/snakeyaml-1.6.jar 2019-11-25T09:29:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/dd-plist-1.20.jar 2019-11-25T09:29:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-4.12.jar 2019-11-25T09:29:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bsh-2.0b6.jar 2019-11-25T09:29:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-launcher-1.0.3.jar 2019-11-25T09:29:36.017 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcifs-1.3.17.jar 2019-11-25T09:29:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/nekohtml-1.9.20.jar 2019-11-25T09:29:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/xercesImpl-2.11.0.jar 2019-11-25T09:29:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jsch-0.1.54.jar 2019-11-25T09:29:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jmespath-java-1.11.267.jar 2019-11-25T09:29:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-databind-2.8.11.jar 2019-11-25T09:29:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-core-2.8.11.jar 2019-11-25T09:29:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jackson-annotations-2.8.11.jar 2019-11-25T09:29:36.018 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/joda-time-2.8.2.jar 2019-11-25T09:29:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/bcprov-jdk15on-1.58.jar 2019-11-25T09:29:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-aether-provider-3.0.4.jar 2019-11-25T09:29:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-impl-1.13.1.jar 2019-11-25T09:29:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-spi-1.13.1.jar 2019-11-25T09:29:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-util-1.13.1.jar 2019-11-25T09:29:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/aether-api-1.13.1.jar 2019-11-25T09:29:36.019 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/wagon-provider-api-2.4.jar 2019-11-25T09:29:36.020 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-classworlds-2.4.jar 2019-11-25T09:29:36.020 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-builder-3.0.4.jar 2019-11-25T09:29:36.020 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-model-3.0.4.jar 2019-11-25T09:29:36.020 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-builder-3.0.4.jar 2019-11-25T09:29:36.020 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-artifact-3.0.4.jar 2019-11-25T09:29:36.020 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-settings-3.0.4.jar 2019-11-25T09:29:36.020 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-sec-dispatcher-1.3.jar 2019-11-25T09:29:36.021 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/maven-repository-metadata-3.0.4.jar 2019-11-25T09:29:36.021 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-utils-3.0.8.jar 2019-11-25T09:29:36.021 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/ion-java-1.0.2.jar 2019-11-25T09:29:36.021 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/httpcore-4.4.9.jar 2019-11-25T09:29:36.021 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/hamcrest-core-1.3.jar 2019-11-25T09:29:36.021 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-engine-1.0.3.jar 2019-11-25T09:29:36.021 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/junit-platform-commons-1.0.3.jar 2019-11-25T09:29:36.022 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/apiguardian-api-1.0.0.jar 2019-11-25T09:29:36.022 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/jcommander-1.47.jar 2019-11-25T09:29:36.022 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-interpolation-1.14.jar 2019-11-25T09:29:36.022 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-component-annotations-1.5.5.jar 2019-11-25T09:29:36.022 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/opentest4j-1.0.0.jar 2019-11-25T09:29:36.022 INFO:teuthology.orchestra.run.smithi153.stdout: inflating: /opt/gradle/gradle-4.7/lib/plugins/plexus-cipher-1.7.jar 2019-11-25T09:29:36.022 INFO:teuthology.orchestra.run.smithi153.stdout:export PATH=/opt/gradle/gradle-4.7/bin:/usr/lib64/qt-3.3/bin:/usr/local/bin:/usr/bin 2019-11-25T09:29:36.690 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:36.690 INFO:teuthology.orchestra.run.smithi153.stdout:Welcome to Gradle 4.7! 2019-11-25T09:29:36.691 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:36.691 INFO:teuthology.orchestra.run.smithi153.stdout:Here are the highlights of this release: 2019-11-25T09:29:36.691 INFO:teuthology.orchestra.run.smithi153.stdout: - Incremental annotation processing 2019-11-25T09:29:36.691 INFO:teuthology.orchestra.run.smithi153.stdout: - JDK 10 support 2019-11-25T09:29:36.691 INFO:teuthology.orchestra.run.smithi153.stdout: - Grouped non-interactive console logs 2019-11-25T09:29:36.691 INFO:teuthology.orchestra.run.smithi153.stdout: - Failed tests are re-run first for quicker feedback 2019-11-25T09:29:36.691 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:36.692 INFO:teuthology.orchestra.run.smithi153.stdout:For more details see https://docs.gradle.org/4.7/release-notes.html 2019-11-25T09:29:36.692 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:37.015 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:37.016 INFO:teuthology.orchestra.run.smithi153.stdout:------------------------------------------------------------ 2019-11-25T09:29:37.016 INFO:teuthology.orchestra.run.smithi153.stdout:Gradle 4.7 2019-11-25T09:29:37.016 INFO:teuthology.orchestra.run.smithi153.stdout:------------------------------------------------------------ 2019-11-25T09:29:37.017 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:37.017 INFO:teuthology.orchestra.run.smithi153.stdout:Build time: 2018-04-18 09:09:12 UTC 2019-11-25T09:29:37.017 INFO:teuthology.orchestra.run.smithi153.stdout:Revision: b9a962bf70638332300e7f810689cb2febbd4a6c 2019-11-25T09:29:37.017 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:37.017 INFO:teuthology.orchestra.run.smithi153.stdout:Groovy: 2.4.12 2019-11-25T09:29:37.017 INFO:teuthology.orchestra.run.smithi153.stdout:Ant: Apache Ant(TM) version 1.9.9 compiled on February 2 2017 2019-11-25T09:29:37.017 INFO:teuthology.orchestra.run.smithi153.stdout:JVM: 1.8.0_232 (Oracle Corporation 25.232-b09) 2019-11-25T09:29:37.018 INFO:teuthology.orchestra.run.smithi153.stdout:OS: Linux 3.10.0-1062.4.3.el7.x86_64 amd64 2019-11-25T09:29:37.018 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:37.021 DEBUG:tasks.s3tests_java:S3 Tests Java: BEGIN 2019-11-25T09:29:37.021 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.a', 'mon.c', 'mgr.y', 'osd.0', 'osd.1', 'osd.2', 'osd.3', 'client.0'] 2019-11-25T09:29:37.022 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi153.front.sepia.ceph.com 2019-11-25T09:29:37.022 DEBUG:tasks.s3tests_java:S3 Tests Java: Cluster config is: ['mon.b', 'mgr.x', 'osd.4', 'osd.5', 'osd.6', 'osd.7', 'client.1'] 2019-11-25T09:29:37.022 DEBUG:tasks.s3tests_java:S3 Tests Java: Host is: ubuntu@smithi081.front.sepia.ceph.com 2019-11-25T09:29:37.022 INFO:tasks.s3tests_java:S3 Tests Java: Creating S3 users... 2019-11-25T09:29:37.454 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf is {'bucket_prefix': 'test-', 's3main': {'access_key': None, 's3dir': 'folder', 'endpoint': None, 'display_name': None, 'region': 'mexico', 'port': None, 'access_secret': None, 'SSE': 'AES256', 'is_secure': None, 'email': None, 'dir': '../data'}} 2019-11-25T09:29:37.455 DEBUG:tasks.s3tests_java:S3 Tests Java: Creating user tester.client.0 2019-11-25T09:29:37.455 DEBUG:tasks.s3tests_java:S3 Tests Java: s3tests_conf[s3main] is {'access_key': 'KPBTIMSRTIDYAGOCBOBA', 's3dir': 'folder', 'endpoint': 'smithi153.front.sepia.ceph.com:80', 'display_name': 'Ms. tester.client.0', 'user_id': 'tester.client.0', 'region': 'mexico', 'port': 80, 'access_secret': '2Jibdpuncjl0DZYJifGabY3VgcVhUGMxI7Udq6SXsphzbcYCTcKRzQ==', 'host': 'smithi153.front.sepia.ceph.com', 'SSE': 'AES256', 'is_secure': False, 'email': 'tester.client.0_test@test.test', 'dir': '../data'} 2019-11-25T09:29:37.455 DEBUG:tasks.s3tests_java:S3 Tests Java: Setion, User = s3main, tester.client.0 2019-11-25T09:29:37.455 INFO:tasks.s3tests_java:['adjust-ulimits', 'ceph-coverage', '/home/ubuntu/cephtest/archive/coverage', 'radosgw-admin', '-n', 'client.0', 'user', 'create', '--uid', 'tester.client.0', '--display-name', 'Ms. tester.client.0', '--access-key', 'KPBTIMSRTIDYAGOCBOBA', '--secret', '2Jibdpuncjl0DZYJifGabY3VgcVhUGMxI7Udq6SXsphzbcYCTcKRzQ==', '--email', 'tester.client.0_test@test.test', '--cluster', 'ceph'] 2019-11-25T09:29:37.456 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid tester.client.0 --display-name 'Ms. tester.client.0' --access-key KPBTIMSRTIDYAGOCBOBA --secret 2Jibdpuncjl0DZYJifGabY3VgcVhUGMxI7Udq6SXsphzbcYCTcKRzQ== --email tester.client.0_test@test.test --cluster ceph 2019-11-25T09:29:37.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.616+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.633 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.646+0000 7f1a56edfec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:37.633 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.646+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.645 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.658+0000 7f1a56edfec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:37.645 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.658+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.667+0000 7f1a56edfec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:37.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.667+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.692+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:37.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.921+0000 7f1a56edfec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:29:37.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.921+0000 7f1a56edfec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:29:37.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.921+0000 7f1a56edfec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:29:37.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.921+0000 7f1a56edfec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:29:37.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.921+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.922 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.935+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:37.922 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.935+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.935 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.948+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:29:37.935 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.948+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.945 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.958+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:37.945 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.958+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.956 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.968+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:29:37.956 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.968+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.961 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.974+0000 7f1a56edfec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:37.961 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.974+0000 7f1a56edfec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:29:37.961 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.974+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.981+0000 7f1a56edfec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:37.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.981+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.991+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:37.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:37.991+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.989 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.002+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:29:37.989 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.002+0000 7f1a56edfec0 20 zonegroup default 2019-11-25T09:29:37.989 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.002+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:37.995 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.008+0000 7f1a56edfec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:37.995 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.008+0000 7f1a56edfec0 10 Cannot find current period zone using local zone 2019-11-25T09:29:37.995 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.008+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:38.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.016+0000 7f1a56edfec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:38.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.016+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:38.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.026+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:38.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.026+0000 7f1a56edfec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:38.028 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.041+0000 7f1a56edfec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:29:38.028 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.041+0000 7f1a56edfec0 20 zone default found 2019-11-25T09:29:38.028 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.041+0000 7f1a56edfec0 20 started sync module instance, tier type = 2019-11-25T09:29:38.028 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.041+0000 7f1a56edfec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:29:38.444 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.457+0000 7f1a56edfec0 20 add_watcher() i=0 2019-11-25T09:29:38.444 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.457+0000 7f1a56edfec0 20 add_watcher() i=1 2019-11-25T09:29:38.444 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.457+0000 7f1a56edfec0 20 add_watcher() i=2 2019-11-25T09:29:38.444 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.457+0000 7f1a56edfec0 20 add_watcher() i=3 2019-11-25T09:29:38.444 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.457+0000 7f1a56edfec0 20 add_watcher() i=4 2019-11-25T09:29:38.444 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.457+0000 7f1a56edfec0 20 add_watcher() i=5 2019-11-25T09:29:38.445 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.457+0000 7f1a56edfec0 20 add_watcher() i=6 2019-11-25T09:29:38.502 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.515+0000 7f1a56edfec0 20 add_watcher() i=7 2019-11-25T09:29:38.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.515+0000 7f1a56edfec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:29:38.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.516+0000 7f1a14fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:29:38.516 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.530+0000 7f1a4410d700 0 -- >> v2:172.21.15.81:3300/0 conn(0x55f514d2bf50 msgr2=0x55f514d30a00 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:29:38.704 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:38.717+0000 7f1a4410d700 0 -- 172.21.15.153:0/2039165272 >> [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] conn(0x55f514d3c530 msgr2=0x55f514d45e60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:29:40.200 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.212+0000 7f1986ffd700 20 reqs_thread_entry: start 2019-11-25T09:29:40.200 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.213+0000 7f1984ff9700 20 reqs_thread_entry: start 2019-11-25T09:29:40.201 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.213+0000 7f1a56edfec0 20 init_complete bucket index max shards: 0 2019-11-25T09:29:40.201 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.213+0000 7f197affd700 20 reqs_thread_entry: start 2019-11-25T09:29:40.201 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.213+0000 7f1a56edfec0 20 get_system_obj_state: rctx=0x7ffc4160cd28 obj=default.rgw.meta:users.uid:tester.client.0 state=0x55f514da6d50 s->prefetch_data=0 2019-11-25T09:29:40.201 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.213+0000 7f1a56edfec0 10 cache get: name=default.rgw.meta+users.uid+tester.client.0 : miss 2019-11-25T09:29:40.209 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.223+0000 7f1a56edfec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x0 2019-11-25T09:29:40.209 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.223+0000 7f1a56edfec0 10 adding default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-11-25T09:29:40.210 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.223+0000 7f1a56edfec0 20 get_system_obj_state: rctx=0x7ffc4160cd38 obj=default.rgw.meta:users.email:tester.client.0_test@test.test state=0x55f514da8ae0 s->prefetch_data=0 2019-11-25T09:29:40.210 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.223+0000 7f1a56edfec0 10 cache get: name=default.rgw.meta+users.email+tester.client.0_test@test.test : miss 2019-11-25T09:29:40.217 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.231+0000 7f1a56edfec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x0 2019-11-25T09:29:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.231+0000 7f1a56edfec0 10 adding default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-11-25T09:29:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.231+0000 7f1a56edfec0 20 get_system_obj_state: rctx=0x7ffc4160cd38 obj=default.rgw.meta:users.keys:KPBTIMSRTIDYAGOCBOBA state=0x55f514da8ae0 s->prefetch_data=0 2019-11-25T09:29:40.218 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.231+0000 7f1a56edfec0 10 cache get: name=default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA : miss 2019-11-25T09:29:40.228 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.237+0000 7f1a56edfec0 10 cache put: name=default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA info.flags=0x0 2019-11-25T09:29:40.228 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.237+0000 7f1a56edfec0 10 adding default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA to cache LRU end 2019-11-25T09:29:40.228 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.237+0000 7f1a56edfec0 20 get_system_obj_state: rctx=0x7ffc4160c948 obj=default.rgw.meta:users.keys:KPBTIMSRTIDYAGOCBOBA state=0x55f514da8ae0 s->prefetch_data=0 2019-11-25T09:29:40.228 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.237+0000 7f1a56edfec0 10 cache get: name=default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA : type miss (requested=0x6, cached=0x0) 2019-11-25T09:29:40.232 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.246+0000 7f1a56edfec0 10 cache put: name=default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA info.flags=0x0 2019-11-25T09:29:40.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.246+0000 7f1a56edfec0 10 moving default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA to cache LRU end 2019-11-25T09:29:40.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.246+0000 7f1a56edfec0 20 get_system_obj_state: rctx=0x7ffc4160cac8 obj=default.rgw.meta:users.keys:KPBTIMSRTIDYAGOCBOBA state=0x55f514da8ae0 s->prefetch_data=0 2019-11-25T09:29:40.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.246+0000 7f1a56edfec0 10 cache get: name=default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA : type miss (requested=0x6, cached=0x0) 2019-11-25T09:29:40.244 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.253+0000 7f1a56edfec0 10 cache put: name=default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA info.flags=0x0 2019-11-25T09:29:40.244 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.253+0000 7f1a56edfec0 10 moving default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA to cache LRU end 2019-11-25T09:29:40.288 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.302+0000 7f1a56edfec0 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-11-25T09:29:40.288 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.302+0000 7f1a56edfec0 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-11-25T09:29:40.289 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.302+0000 7f1a56edfec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=632 2019-11-25T09:29:40.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.317+0000 7f1a39908700 10 RGWWatcher::handle_notify() notify_id 579820584962 cookie 94511104623376 notifier 4505 bl.length()=632 2019-11-25T09:29:40.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.317+0000 7f1a39908700 10 cache put: name=default.rgw.meta+users.uid+tester.client.0 info.flags=0x17 2019-11-25T09:29:40.304 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.317+0000 7f1a39908700 10 moving default.rgw.meta+users.uid+tester.client.0 to cache LRU end 2019-11-25T09:29:40.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.871+0000 7f1a56edfec0 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-11-25T09:29:40.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.871+0000 7f1a56edfec0 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-11-25T09:29:40.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.871+0000 7f1a56edfec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=200 2019-11-25T09:29:40.876 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.890+0000 7f1a39908700 10 RGWWatcher::handle_notify() notify_id 579820584966 cookie 94511104615104 notifier 4505 bl.length()=200 2019-11-25T09:29:40.877 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.890+0000 7f1a39908700 10 cache put: name=default.rgw.meta+users.email+tester.client.0_test@test.test info.flags=0x7 2019-11-25T09:29:40.877 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:40.890+0000 7f1a39908700 10 moving default.rgw.meta+users.email+tester.client.0_test@test.test to cache LRU end 2019-11-25T09:29:41.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.331+0000 7f1a56edfec0 10 cache put: name=default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA info.flags=0x7 2019-11-25T09:29:41.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.331+0000 7f1a56edfec0 10 moving default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA to cache LRU end 2019-11-25T09:29:41.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.331+0000 7f1a56edfec0 10 distributing notification oid=default.rgw.control:notify.3 bl.length()=189 2019-11-25T09:29:41.325 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.339+0000 7f1a39908700 10 RGWWatcher::handle_notify() notify_id 579820584967 cookie 94511104615104 notifier 4505 bl.length()=189 2019-11-25T09:29:41.326 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.339+0000 7f1a39908700 10 cache put: name=default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA info.flags=0x7 2019-11-25T09:29:41.326 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.339+0000 7f1a39908700 10 moving default.rgw.meta+users.keys+KPBTIMSRTIDYAGOCBOBA to cache LRU end 2019-11-25T09:29:41.394 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2019-11-25T09:29:41.394 INFO:teuthology.orchestra.run.smithi153.stdout: "user_id": "tester.client.0", 2019-11-25T09:29:41.394 INFO:teuthology.orchestra.run.smithi153.stdout: "display_name": "Ms. tester.client.0", 2019-11-25T09:29:41.394 INFO:teuthology.orchestra.run.smithi153.stdout: "email": "tester.client.0_test@test.test", 2019-11-25T09:29:41.394 INFO:teuthology.orchestra.run.smithi153.stdout: "suspended": 0, 2019-11-25T09:29:41.394 INFO:teuthology.orchestra.run.smithi153.stdout: "max_buckets": 1000, 2019-11-25T09:29:41.395 INFO:teuthology.orchestra.run.smithi153.stdout: "subusers": [], 2019-11-25T09:29:41.395 INFO:teuthology.orchestra.run.smithi153.stdout: "keys": [ 2019-11-25T09:29:41.395 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T09:29:41.395 INFO:teuthology.orchestra.run.smithi153.stdout: "user": "tester.client.0", 2019-11-25T09:29:41.395 INFO:teuthology.orchestra.run.smithi153.stdout: "access_key": "KPBTIMSRTIDYAGOCBOBA", 2019-11-25T09:29:41.395 INFO:teuthology.orchestra.run.smithi153.stdout: "secret_key": "2Jibdpuncjl0DZYJifGabY3VgcVhUGMxI7Udq6SXsphzbcYCTcKRzQ==" 2019-11-25T09:29:41.395 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T09:29:41.396 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T09:29:41.396 INFO:teuthology.orchestra.run.smithi153.stdout: "swift_keys": [], 2019-11-25T09:29:41.396 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": [], 2019-11-25T09:29:41.396 INFO:teuthology.orchestra.run.smithi153.stdout: "op_mask": "read, write, delete", 2019-11-25T09:29:41.396 INFO:teuthology.orchestra.run.smithi153.stdout: "default_placement": "", 2019-11-25T09:29:41.396 INFO:teuthology.orchestra.run.smithi153.stdout: "default_storage_class": "", 2019-11-25T09:29:41.397 INFO:teuthology.orchestra.run.smithi153.stdout: "placement_tags": [], 2019-11-25T09:29:41.397 INFO:teuthology.orchestra.run.smithi153.stdout: "bucket_quota": { 2019-11-25T09:29:41.397 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T09:29:41.397 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T09:29:41.398 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T09:29:41.398 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T09:29:41.398 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T09:29:41.398 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T09:29:41.398 INFO:teuthology.orchestra.run.smithi153.stdout: "user_quota": { 2019-11-25T09:29:41.398 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T09:29:41.398 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T09:29:41.398 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T09:29:41.399 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T09:29:41.399 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T09:29:41.399 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T09:29:41.399 INFO:teuthology.orchestra.run.smithi153.stdout: "temp_url_keys": [], 2019-11-25T09:29:41.399 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "rgw", 2019-11-25T09:29:41.399 INFO:teuthology.orchestra.run.smithi153.stdout: "mfa_ids": [] 2019-11-25T09:29:41.406 INFO:teuthology.orchestra.run.smithi153.stdout:} 2019-11-25T09:29:41.406 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:29:41.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.447+0000 7f1a56edfec0 20 remove_watcher() i=0 2019-11-25T09:29:41.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.447+0000 7f1a56edfec0 2 removed watcher, disabling cache 2019-11-25T09:29:41.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.503+0000 7f1a56edfec0 20 remove_watcher() i=1 2019-11-25T09:29:41.549 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.560+0000 7f1a56edfec0 20 remove_watcher() i=2 2019-11-25T09:29:41.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.598+0000 7f1a56edfec0 20 remove_watcher() i=3 2019-11-25T09:29:41.626 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.639+0000 7f1a56edfec0 20 remove_watcher() i=4 2019-11-25T09:29:41.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.677+0000 7f1a56edfec0 20 remove_watcher() i=5 2019-11-25T09:29:41.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.708+0000 7f1a56edfec0 20 remove_watcher() i=6 2019-11-25T09:29:41.722 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:41.736+0000 7f1a56edfec0 20 remove_watcher() i=7 2019-11-25T09:29:41.729 INFO:teuthology.orchestra.run.smithi153:> cat > /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf 2019-11-25T09:29:41.783 INFO:tasks.s3tests_java:S3 Tests Java: Running tests... 2019-11-25T09:29:41.784 INFO:teuthology.orchestra.run.smithi153:> cp /home/ubuntu/cephtest/archive/s3-tests-java.client.0.conf /home/ubuntu/cephtest/s3-tests-java/config.properties 2019-11-25T09:29:41.862 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:29:42.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.064+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.085 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.098+0000 7fe932be0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:42.085 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.098+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.097 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.111+0000 7fe932be0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:42.098 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.111+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.117+0000 7fe932be0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:42.104 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.117+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.128 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.141+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:42.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.362+0000 7fe932be0ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:29:42.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.362+0000 7fe932be0ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:29:42.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.362+0000 7fe932be0ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:29:42.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.362+0000 7fe932be0ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:29:42.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.362+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.378+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:42.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.378+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.398+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:29:42.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.398+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.418+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:42.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.418+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.440+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:29:42.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.440+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.438 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.451+0000 7fe932be0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:42.438 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.451+0000 7fe932be0ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:29:42.438 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.451+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.449 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.463+0000 7fe932be0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:42.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.463+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.479+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:42.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.479+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.496+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:29:42.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.496+0000 7fe932be0ec0 20 zonegroup default 2019-11-25T09:29:42.483 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.496+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.494 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.508+0000 7fe932be0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:42.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.508+0000 7fe932be0ec0 10 Cannot find current period zone using local zone 2019-11-25T09:29:42.496 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.508+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.517+0000 7fe932be0ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:42.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.517+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.516 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.530+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:42.516 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.530+0000 7fe932be0ec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:42.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.539+0000 7fe932be0ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:29:42.527 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.539+0000 7fe932be0ec0 20 zone default found 2019-11-25T09:29:42.527 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.539+0000 7fe932be0ec0 20 started sync module instance, tier type = 2019-11-25T09:29:42.527 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.539+0000 7fe932be0ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:29:42.927 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.940+0000 7fe932be0ec0 20 add_watcher() i=0 2019-11-25T09:29:42.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.940+0000 7fe932be0ec0 20 add_watcher() i=1 2019-11-25T09:29:42.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.940+0000 7fe932be0ec0 20 add_watcher() i=2 2019-11-25T09:29:42.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.940+0000 7fe932be0ec0 20 add_watcher() i=3 2019-11-25T09:29:42.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.940+0000 7fe932be0ec0 20 add_watcher() i=4 2019-11-25T09:29:42.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.940+0000 7fe932be0ec0 20 add_watcher() i=5 2019-11-25T09:29:42.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.940+0000 7fe932be0ec0 20 add_watcher() i=6 2019-11-25T09:29:42.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.982+0000 7fe932be0ec0 20 add_watcher() i=7 2019-11-25T09:29:42.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.982+0000 7fe932be0ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:29:42.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.982+0000 7fe873fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:29:42.976 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:42.990+0000 7fe91fe0e700 0 -- 172.21.15.153:0/3869247969 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcd6feb0d0 msgr2=0x55dcd6feb4e0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:29:43.179 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:43.193+0000 7fe91fe0e700 0 --2- 172.21.15.153:0/3869247969 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55dcd6feb0d0 0x55dcd6feb4e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-25T09:29:43.227 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:43.240+0000 7fe91fe0e700 0 -- 172.21.15.153:0/4250913136 >> v2:172.21.15.81:3300/0 conn(0x55dcd6fe61f0 msgr2=0x55dcd6fe6600 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:29:44.652 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.664+0000 7fe85e7fc700 20 reqs_thread_entry: start 2019-11-25T09:29:44.652 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.664+0000 7fe857fff700 20 reqs_thread_entry: start 2019-11-25T09:29:44.652 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.665+0000 7fe932be0ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:29:44.653 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.665+0000 7fe8567fc700 20 reqs_thread_entry: start 2019-11-25T09:29:44.653 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.665+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:29:44.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.743+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:44.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.763+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:44.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.791+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:29:44.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.791+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.85', time=2019-11-25T11:29:17.872297+0000, chain.objs.size()=3 2019-11-25T09:29:44.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.791+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6__shadow_.D0Shh7f9Qqb0mnSWX6B48-oP0G4cSbv_0 2019-11-25T09:29:44.779 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.791+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6__shadow_.D0Shh7f9Qqb0mnSWX6B48-oP0G4cSbv_1 2019-11-25T09:29:44.779 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.791+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6__shadow_.D0Shh7f9Qqb0mnSWX6B48-oP0G4cSbv_2 2019-11-25T09:29:44.935 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:44.948+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:45.095 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.109+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:29:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.175+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:45.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.203+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:45.230 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.242+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:29:45.230 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.242+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.225', time=2019-11-25T11:29:11.923197+0000, chain.objs.size()=5 2019-11-25T09:29:45.231 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.242+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.1 2019-11-25T09:29:45.231 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.242+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.1_1 2019-11-25T09:29:45.231 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.242+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.2 2019-11-25T09:29:45.231 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.242+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.2_1 2019-11-25T09:29:45.231 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.242+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.3 2019-11-25T09:29:45.231 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.243+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_MbZUKTZE1RYYKYtajIXiNUvClsmkPTi', time=2019-11-25T11:29:12.098496+0000, chain.objs.size()=3 2019-11-25T09:29:45.232 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.243+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.L2RHLGke6rI_5_NnXhgzQo5soK72LJo_0 2019-11-25T09:29:45.232 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.243+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.L2RHLGke6rI_5_NnXhgzQo5soK72LJo_1 2019-11-25T09:29:45.232 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.243+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.L2RHLGke6rI_5_NnXhgzQo5soK72LJo_2 2019-11-25T09:29:45.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.467+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:45.615 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.629+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:29:45.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.712+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:45.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.739+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:45.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:29:45.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.233', time=2019-11-25T11:29:12.499837+0000, chain.objs.size()=5 2019-11-25T09:29:45.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.1 2019-11-25T09:29:45.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.1_1 2019-11-25T09:29:45.774 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.2 2019-11-25T09:29:45.774 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.2_1 2019-11-25T09:29:45.774 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.3 2019-11-25T09:29:45.774 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.235', time=2019-11-25T11:29:12.971729+0000, chain.objs.size()=5 2019-11-25T09:29:45.774 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.1 2019-11-25T09:29:45.774 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.1_1 2019-11-25T09:29:45.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.2 2019-11-25T09:29:45.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.2_1 2019-11-25T09:29:45.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.786+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-LUKEWARM.2~lBthdRFcGMwVuumOrEai28fna72_Atc.3 2019-11-25T09:29:45.939 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:45.953+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:46.025 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.038+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:29:46.068 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.082+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:46.093 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.101+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:46.138 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=4, truncated=0, next_marker='' 2019-11-25T09:29:46.140 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_J714CAhrvbaK4jNY3N7AIx2hlfhYvyn', time=2019-11-25T11:29:12.296308+0000, chain.objs.size()=2 2019-11-25T09:29:46.140 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.kRpDnTjIzYbk3iBreImGIm-B36Ge8Ha_1 2019-11-25T09:29:46.140 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.kRpDnTjIzYbk3iBreImGIm-B36Ge8Ha_2 2019-11-25T09:29:46.140 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_DmCHlKT-MIVQ2KWqL-HE0UI4MRIlbiI', time=2019-11-25T11:29:12.726116+0000, chain.objs.size()=3 2019-11-25T09:29:46.140 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.3e4ypJmMuGhwcxLK43aBPkMX6Rnv3ZU_0 2019-11-25T09:29:46.140 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.3e4ypJmMuGhwcxLK43aBPkMX6Rnv3ZU_1 2019-11-25T09:29:46.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.3e4ypJmMuGhwcxLK43aBPkMX6Rnv3ZU_2 2019-11-25T09:29:46.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.245', time=2019-11-25T11:29:14.135025+0000, chain.objs.size()=3 2019-11-25T09:29:46.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-STANDARD.2~p6D2O6DCk-cRLhfe0Ixn5JUKzbnFt0i.1 2019-11-25T09:29:46.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-STANDARD.2~p6D2O6DCk-cRLhfe0Ixn5JUKzbnFt0i.2 2019-11-25T09:29:46.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-STANDARD.2~p6D2O6DCk-cRLhfe0Ixn5JUKzbnFt0i.3 2019-11-25T09:29:46.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.196', time=2019-11-25T11:29:15.653099+0000, chain.objs.size()=5 2019-11-25T09:29:46.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-FROZEN.2~85dlfIbenR3hCJ8S5OYeclfDBZjTa1M.1 2019-11-25T09:29:46.142 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__shadow_foo-FROZEN.2~85dlfIbenR3hCJ8S5OYeclfDBZjTa1M.1_1 2019-11-25T09:29:46.142 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-FROZEN.2~85dlfIbenR3hCJ8S5OYeclfDBZjTa1M.2 2019-11-25T09:29:46.142 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.150+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__shadow_foo-FROZEN.2~85dlfIbenR3hCJ8S5OYeclfDBZjTa1M.2_1 2019-11-25T09:29:46.369 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.383+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-FROZEN.2~85dlfIbenR3hCJ8S5OYeclfDBZjTa1M.3 2019-11-25T09:29:46.554 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.568+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:46.676 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.690+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:29:46.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.741+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:46.745 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.758+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:46.754 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.768+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:46.799 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.813+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:29:46.872 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.885+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:46.902 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.916+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:29:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.20', time=2019-11-25T11:29:01.193900+0000, chain.objs.size()=7 2019-11-25T09:29:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2__multipart_foo.2~nFpujveOA1sp5iM66Mu7OmS3cSSsCD1.1 2019-11-25T09:29:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2__shadow_foo.2~nFpujveOA1sp5iM66Mu7OmS3cSSsCD1.1_1 2019-11-25T09:29:46.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2__multipart_foo.2~nFpujveOA1sp5iM66Mu7OmS3cSSsCD1.2 2019-11-25T09:29:46.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2__shadow_foo.2~nFpujveOA1sp5iM66Mu7OmS3cSSsCD1.2_1 2019-11-25T09:29:46.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2__multipart_foo.2~nFpujveOA1sp5iM66Mu7OmS3cSSsCD1.3 2019-11-25T09:29:46.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2__shadow_foo.2~nFpujveOA1sp5iM66Mu7OmS3cSSsCD1.3_1 2019-11-25T09:29:46.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:46.967+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.2__multipart_foo.2~nFpujveOA1sp5iM66Mu7OmS3cSSsCD1.4 2019-11-25T09:29:47.337 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.351+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:47.455 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.468+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:29:47.511 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.525+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:47.535 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.549+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:47.549 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.562+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:47.617 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.631+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:29:47.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.711+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:47.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.756+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:47.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.774+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:47.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.827+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:29:47.885 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.898+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:47.916 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.929+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:47.934 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.948+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:47.973 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:47.986+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:29:48.041 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.054+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:48.068 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.081+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:48.089 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.102+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:29:48.089 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.102+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_QtjwwJWyokczo1QPHbAgGUKxZ7dRYkr', time=2019-11-25T11:29:08.116517+0000, chain.objs.size()=1 2019-11-25T09:29:48.089 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.102+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.B-_7bYOFUba1d5tXQy_xwHM363UFHXj_1 2019-11-25T09:29:48.089 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.102+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_Pug5muhZYUp9Sg8KtEVANLcFSNBGpgf', time=2019-11-25T11:29:13.241964+0000, chain.objs.size()=2 2019-11-25T09:29:48.089 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.102+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.anA4VLhT243u4uJVeD6IEf5Bg2aS8Wi_1 2019-11-25T09:29:48.089 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.102+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.anA4VLhT243u4uJVeD6IEf5Bg2aS8Wi_2 2019-11-25T09:29:48.241 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.255+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:48.312 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.326+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:29:48.359 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.372+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:48.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.390+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:48.387 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.400+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:48.423 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.436+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:29:48.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.498+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:48.505 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.519+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:48.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.550+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:29:48.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.550+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.108', time=2019-11-25T11:29:07.590177+0000, chain.objs.size()=2 2019-11-25T09:29:48.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.550+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.T9eQHlDwYa5ax8MnE8uJZK7QsZMS0ZM_0 2019-11-25T09:29:48.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.550+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.T9eQHlDwYa5ax8MnE8uJZK7QsZMS0ZM_1 2019-11-25T09:29:48.569 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.583+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:48.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.679+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:29:48.758 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.766+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:48.783 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.796+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:48.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.837+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:29:48.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.837+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.40', time=2019-11-25T11:29:03.119652+0000, chain.objs.size()=4 2019-11-25T09:29:48.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.837+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3__multipart_foo.2~lX1vkTotxpt76RiH1__rB18fBceccDc.1 2019-11-25T09:29:48.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.837+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3__multipart_foo.2~lX1vkTotxpt76RiH1__rB18fBceccDc.2 2019-11-25T09:29:48.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.837+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3__multipart_foo.2~lX1vkTotxpt76RiH1__rB18fBceccDc.3 2019-11-25T09:29:48.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.837+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.3__multipart_foo.2~lX1vkTotxpt76RiH1__rB18fBceccDc.4 2019-11-25T09:29:48.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.837+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_YGx0-T4jxziGGdW1NympQs240w1eB1L', time=2019-11-25T11:29:09.932142+0000, chain.objs.size()=1 2019-11-25T09:29:48.826 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:48.837+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.7dsQTfOGw1ZXCkgITEuZuJoXNdhpgsX_0 2019-11-25T09:29:49.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.068+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:49.171 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.184+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:29:49.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.260+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:49.306 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.319+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:49.320 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.334+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:49.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.379+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:29:49.421 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.434+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:49.455 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.468+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:49.467 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.480+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:49.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.521+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:29:49.570 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.584+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:49.595 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.608+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:49.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.621+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:49.671 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.685+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:29:49.719 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.732+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:49.734 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.748+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:49.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:29:49.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.210', time=2019-11-25T11:29:16.141629+0000, chain.objs.size()=5 2019-11-25T09:29:49.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-STANDARD.2~JNJReHwbcZekWIu7icMuJ8HsPnAo5FU.1 2019-11-25T09:29:49.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__shadow_foo-STANDARD.2~JNJReHwbcZekWIu7icMuJ8HsPnAo5FU.1_1 2019-11-25T09:29:49.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-STANDARD.2~JNJReHwbcZekWIu7icMuJ8HsPnAo5FU.2 2019-11-25T09:29:49.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__shadow_foo-STANDARD.2~JNJReHwbcZekWIu7icMuJ8HsPnAo5FU.2_1 2019-11-25T09:29:49.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-STANDARD.2~JNJReHwbcZekWIu7icMuJ8HsPnAo5FU.3 2019-11-25T09:29:49.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.84', time=2019-11-25T11:29:17.630726+0000, chain.objs.size()=3 2019-11-25T09:29:49.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6__shadow_.XLx8a318-F1CLV215iYPzOjBVkCWPh-_0 2019-11-25T09:29:49.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6__shadow_.XLx8a318-F1CLV215iYPzOjBVkCWPh-_1 2019-11-25T09:29:49.751 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:49.762+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.6__shadow_.XLx8a318-F1CLV215iYPzOjBVkCWPh-_2 2019-11-25T09:29:50.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.054+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:50.152 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.165+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:29:50.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.199+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:50.221 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.235+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:50.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:29:50.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.114', time=2019-11-25T11:29:08.847119+0000, chain.objs.size()=2 2019-11-25T09:29:50.237 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.MctkdB13wzrzOh8L8cd8PVhCmzgio56_0 2019-11-25T09:29:50.237 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.MctkdB13wzrzOh8L8cd8PVhCmzgio56_1 2019-11-25T09:29:50.237 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.224', time=2019-11-25T11:29:11.706724+0000, chain.objs.size()=5 2019-11-25T09:29:50.237 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.1 2019-11-25T09:29:50.237 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.1_1 2019-11-25T09:29:50.237 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.2 2019-11-25T09:29:50.238 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.2_1 2019-11-25T09:29:50.238 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.249+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-FROZEN.2~NW3vb0tNSICUrjVdYvzlR_RbawAT84r.3 2019-11-25T09:29:50.352 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.366+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:50.421 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.434+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:29:50.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.495+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:50.505 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.518+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:50.516 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.529+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:29:50.516 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.529+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_hSJ0aEf3SOBpPFeJmfGzmptgAKwzJrr', time=2019-11-25T11:29:08.602682+0000, chain.objs.size()=2 2019-11-25T09:29:50.516 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.529+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_._54AOv4iz4tMjlG7T5dZbQ6P3LjtGXf_0 2019-11-25T09:29:50.516 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.529+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_._54AOv4iz4tMjlG7T5dZbQ6P3LjtGXf_1 2019-11-25T09:29:50.572 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.586+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:50.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.674+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:29:50.746 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.759+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:50.783 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.796+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:50.799 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.813+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:50.862 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.876+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:29:50.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.942+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:50.952 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.966+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:50.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.981+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:29:50.968 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.981+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_j2oy8VDr3xbb__AzO3-dyuQ9R1QbZQJ', time=2019-11-25T11:29:09.053929+0000, chain.objs.size()=1 2019-11-25T09:29:50.968 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.981+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.CMxRCstJ8mm6MNIBPCHz-uEAGhjvn8P_1 2019-11-25T09:29:50.968 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.981+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_rNr6UH_m01Fwy_PokqYmu8J0fE91Tf9', time=2019-11-25T11:29:13.920418+0000, chain.objs.size()=1 2019-11-25T09:29:50.968 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:50.981+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.eZMtcm1iSzVRKSou7IaEsq83vd1QyxV_0 2019-11-25T09:29:51.059 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.072+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:51.154 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.168+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:29:51.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.227+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:51.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.247+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:51.243 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.256+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:51.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.324+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:29:51.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.379+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:51.387 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.401+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:51.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.418+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:29:51.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.418+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.112', time=2019-11-25T11:29:08.371618+0000, chain.objs.size()=2 2019-11-25T09:29:51.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.418+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.MctkdB13wzrzOh8L8cd8PVhCmzgio56_0 2019-11-25T09:29:51.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.418+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.MctkdB13wzrzOh8L8cd8PVhCmzgio56_1 2019-11-25T09:29:51.406 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.418+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_14ycp4Su54dYzT5L1sJurUdxz-Q2YXy', time=2019-11-25T11:29:09.658848+0000, chain.objs.size()=1 2019-11-25T09:29:51.406 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.418+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.Th0pnC4QwrRyDxhxvutfUjCxn_A30as_0 2019-11-25T09:29:51.500 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.513+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:51.601 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.615+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:29:51.674 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.688+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:51.694 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.707+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:51.705 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.719+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:51.745 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.759+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:29:51.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.823+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:51.830 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.844+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:51.850 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-11-25T09:29:51.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.55', time=2019-11-25T11:29:04.824820+0000, chain.objs.size()=4 2019-11-25T09:29:51.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4__multipart_foo.2~QhK2O8B9Sia255VBtxN0UpvkaObaWA1.1 2019-11-25T09:29:51.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4__multipart_foo.2~QhK2O8B9Sia255VBtxN0UpvkaObaWA1.2 2019-11-25T09:29:51.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4__multipart_foo.2~QhK2O8B9Sia255VBtxN0UpvkaObaWA1.3 2019-11-25T09:29:51.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.4__multipart_foo.2~QhK2O8B9Sia255VBtxN0UpvkaObaWA1.4 2019-11-25T09:29:51.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_U9K2jFSjDc7h8eIerIrYpfHDoqAK14x', time=2019-11-25T11:29:07.939462+0000, chain.objs.size()=2 2019-11-25T09:29:51.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.3kDEx0aGM5kBZXeiYKMtcQFx5a9VvE3_0 2019-11-25T09:29:51.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.3kDEx0aGM5kBZXeiYKMtcQFx5a9VvE3_1 2019-11-25T09:29:51.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.242', time=2019-11-25T11:29:13.460045+0000, chain.objs.size()=3 2019-11-25T09:29:51.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-STANDARD.2~p6D2O6DCk-cRLhfe0Ixn5JUKzbnFt0i.1 2019-11-25T09:29:51.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-STANDARD.2~p6D2O6DCk-cRLhfe0Ixn5JUKzbnFt0i.2 2019-11-25T09:29:51.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:51.863+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__multipart_foo-STANDARD.2~p6D2O6DCk-cRLhfe0Ixn5JUKzbnFt0i.3 2019-11-25T09:29:52.097 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.110+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:52.162 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.176+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:29:52.202 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.216+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:52.218 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.232+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:52.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.242+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:29:52.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.242+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='_R_VLqKNLl8LO1nzd_y1Jb3LhdSWbobE', time=2019-11-25T11:29:13.649688+0000, chain.objs.size()=1 2019-11-25T09:29:52.237 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.242+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.9__shadow_.fnYWXMxoTygb3war4cpwHn9FRI1Qhbd_0 2019-11-25T09:29:52.305 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.318+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:52.375 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.389+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:29:52.456 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.470+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:52.491 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.503+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:52.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.528+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:29:52.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.528+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.203', time=2019-11-25T11:29:15.918027+0000, chain.objs.size()=5 2019-11-25T09:29:52.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.528+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-LUKEWARM.2~jlWWI_aOxsfkkm5WxgnBzYd9iBYk-p6.1 2019-11-25T09:29:52.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.528+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__shadow_foo-LUKEWARM.2~jlWWI_aOxsfkkm5WxgnBzYd9iBYk-p6.1_1 2019-11-25T09:29:52.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.528+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-LUKEWARM.2~jlWWI_aOxsfkkm5WxgnBzYd9iBYk-p6.2 2019-11-25T09:29:52.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.528+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__shadow_foo-LUKEWARM.2~jlWWI_aOxsfkkm5WxgnBzYd9iBYk-p6.2_1 2019-11-25T09:29:52.516 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.528+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.lukewarm:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.8__multipart_foo-LUKEWARM.2~jlWWI_aOxsfkkm5WxgnBzYd9iBYk-p6.3 2019-11-25T09:29:52.664 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.677+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:52.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.774+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:29:52.844 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.857+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:52.870 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.884+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:52.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.905+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:29:52.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.905+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.109', time=2019-11-25T11:29:07.776084+0000, chain.objs.size()=2 2019-11-25T09:29:52.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.905+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.T9eQHlDwYa5ax8MnE8uJZK7QsZMS0ZM_0 2019-11-25T09:29:52.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.905+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data.frozen:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.7__shadow_.T9eQHlDwYa5ax8MnE8uJZK7QsZMS0ZM_1 2019-11-25T09:29:52.957 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:52.970+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:53.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.108+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:29:53.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.172+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:53.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.200+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:53.205 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.219+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:53.264 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.277+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:29:53.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.346+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:53.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.375+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:53.375 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.389+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:53.424 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.437+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:29:53.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.591+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:53.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.614+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:53.611 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.625+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:53.649 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.662+0000 7fe932be0ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:29:53.721 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.734+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:53.746 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.760+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:53.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.773+0000 7fe932be0ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:29:53.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.773+0000 7fe932be0ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.71', time=2019-11-25T11:29:06.782206+0000, chain.objs.size()=4 2019-11-25T09:29:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.773+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5__multipart_foo.2~D9pp9OnqiCJ4HlXGFplCImOlLqMjKYA.1 2019-11-25T09:29:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.773+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5__multipart_foo.2~D9pp9OnqiCJ4HlXGFplCImOlLqMjKYA.2 2019-11-25T09:29:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.773+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5__multipart_foo.2~D9pp9OnqiCJ4HlXGFplCImOlLqMjKYA.3 2019-11-25T09:29:53.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.773+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.5__multipart_foo.2~D9pp9OnqiCJ4HlXGFplCImOlLqMjKYA.4 2019-11-25T09:29:53.951 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:53.963+0000 7fe932be0ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:29:54.090 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.104+0000 7fe932be0ec0 20 remove_watcher() i=0 2019-11-25T09:29:54.090 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.104+0000 7fe932be0ec0 2 removed watcher, disabling cache 2019-11-25T09:29:54.139 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.152+0000 7fe932be0ec0 20 remove_watcher() i=1 2019-11-25T09:29:54.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.199+0000 7fe932be0ec0 20 remove_watcher() i=2 2019-11-25T09:29:54.250 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.264+0000 7fe932be0ec0 20 remove_watcher() i=3 2019-11-25T09:29:54.301 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.315+0000 7fe932be0ec0 20 remove_watcher() i=4 2019-11-25T09:29:54.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.358+0000 7fe932be0ec0 20 remove_watcher() i=5 2019-11-25T09:29:54.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.397+0000 7fe932be0ec0 20 remove_watcher() i=6 2019-11-25T09:29:54.418 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.432+0000 7fe932be0ec0 20 remove_watcher() i=7 2019-11-25T09:29:54.425 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:29:54.607 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.620+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:54.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.648+0000 7f01ad68aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:54.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.648+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:54.644 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.658+0000 7f01ad68aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:54.644 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.658+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:54.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.668+0000 7f01ad68aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:54.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.668+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:54.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.693+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:54.931 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.944+0000 7f01ad68aec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:29:54.931 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.944+0000 7f01ad68aec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:29:54.931 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.944+0000 7f01ad68aec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:29:54.932 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.944+0000 7f01ad68aec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:29:54.932 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.944+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:54.952 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.966+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:54.953 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.966+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:54.966 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.980+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:29:54.966 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.980+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:54.976 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.990+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:54.977 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:54.990+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:54.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.004+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:29:54.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.004+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:55.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.016+0000 7f01ad68aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:55.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.016+0000 7f01ad68aec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:29:55.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.016+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:55.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.026+0000 7f01ad68aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:55.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.026+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:55.027 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.041+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:55.027 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.041+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:55.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.056+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:29:55.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.056+0000 7f01ad68aec0 20 zonegroup default 2019-11-25T09:29:55.043 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.056+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:55.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.074+0000 7f01ad68aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:55.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.074+0000 7f01ad68aec0 10 Cannot find current period zone using local zone 2019-11-25T09:29:55.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.074+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:55.068 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.082+0000 7f01ad68aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:29:55.069 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.082+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:55.082 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.095+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:29:55.082 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.095+0000 7f01ad68aec0 20 rados->read ofs=0 len=0 2019-11-25T09:29:55.090 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.104+0000 7f01ad68aec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:29:55.090 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.104+0000 7f01ad68aec0 20 zone default found 2019-11-25T09:29:55.090 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.104+0000 7f01ad68aec0 20 started sync module instance, tier type = 2019-11-25T09:29:55.090 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.104+0000 7f01ad68aec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:29:55.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.417+0000 7f01ad68aec0 20 add_watcher() i=0 2019-11-25T09:29:55.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.417+0000 7f01ad68aec0 20 add_watcher() i=1 2019-11-25T09:29:55.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.417+0000 7f01ad68aec0 20 add_watcher() i=2 2019-11-25T09:29:55.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.417+0000 7f01ad68aec0 20 add_watcher() i=3 2019-11-25T09:29:55.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.417+0000 7f01ad68aec0 20 add_watcher() i=4 2019-11-25T09:29:55.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.417+0000 7f01ad68aec0 20 add_watcher() i=5 2019-11-25T09:29:55.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.417+0000 7f01ad68aec0 20 add_watcher() i=6 2019-11-25T09:29:55.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.445+0000 7f01ad68aec0 20 add_watcher() i=7 2019-11-25T09:29:55.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.445+0000 7f01ad68aec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:29:55.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.445+0000 7f016b7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:29:55.434 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.448+0000 7f019a0b7700 0 -- 172.21.15.153:0/2165762709 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564341c503f0 msgr2=0x564341c3c120 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:29:55.474 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:55.487+0000 7f019a8b8700 0 -- 172.21.15.153:0/2620519421 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7f00dc017580 msgr2=0x7f00dc019a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:29:57.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.270+0000 7f00d9ffb700 20 reqs_thread_entry: start 2019-11-25T09:29:57.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.271+0000 7f00d37fe700 20 reqs_thread_entry: start 2019-11-25T09:29:57.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.271+0000 7f01ad68aec0 20 init_complete bucket index max shards: 0 2019-11-25T09:29:57.258 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.272+0000 7f00d1ffb700 20 reqs_thread_entry: start 2019-11-25T09:29:57.258 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.272+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:29:57.336 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.350+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.375+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:57.374 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.387+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.423 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.437+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:29:57.473 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.487+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.493 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.506+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:57.505 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.519+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.560+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:29:57.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.622+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.627 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.640+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:57.635 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.649+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.673 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.687+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:29:57.711 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.725+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.743+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:57.738 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.752+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.767 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.780+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:29:57.819 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.832+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.848 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.861+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:57.871 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.885+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.934 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.947+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:29:57.979 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:57.992+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:57.994 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.008+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:58.002 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.016+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.057 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.070+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:29:58.110 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.123+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.135 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.149+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:58.146 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.160+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.199+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:29:58.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.261+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.288+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:58.289 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.302+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.344 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.357+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:29:58.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.443+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.462 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.475+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:58.475 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.489+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.582+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:29:58.635 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.649+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.669 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.683+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:58.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.698+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.739+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:29:58.784 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.798+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.812 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.826+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:58.828 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.841+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:58.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.904+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:29:58.970 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:58.984+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.015+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:59.016 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.030+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.084 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.097+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:29:59.162 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.175+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.207+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:59.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.219+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.267+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:29:59.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.354+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.370 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.383+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:59.386 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.400+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.494+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:29:59.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.581+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.597 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.610+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:59.618 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.632+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.680+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:29:59.722 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.735+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.754 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.768+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:59.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.783+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.823+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:29:59.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.872+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.875 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.889+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:59.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.900+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.921 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.935+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:29:59.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.981+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:29:59.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:29:59.998+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:29:59.993 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.007+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.026 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.040+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:30:00.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.117+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.133 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.147+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:00.148 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.162+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.200 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.214+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:30:00.281 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.284+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.294 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.307+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:00.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.321+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.347 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.360+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:30:00.419 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.432+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.443 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.456+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:00.464 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.475+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.510 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.524+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:30:00.586 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.600+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.630+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:00.632 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.646+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.696+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:30:00.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.757+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.766 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.780+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.793+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.827+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:30:00.895 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.909+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:00.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.958+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:00.958 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:00.972+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.025+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:30:01.097 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.110+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.122 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.136+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:01.135 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.149+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.173 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.187+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:30:01.248 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.262+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.278 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.292+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:01.290 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.304+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.346+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:30:01.390 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.404+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.408 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.422+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:01.421 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.435+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.480+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:30:01.536 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.549+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.561 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.575+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:01.573 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.587+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.754 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.768+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:30:01.829 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.843+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.849 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.863+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:01.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.876+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.922+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:30:01.961 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:01.975+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:01.986 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.000+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:01.998 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.012+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:02.050 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.063+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:30:02.123 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.137+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:02.152 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.166+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:02.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.181+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:02.210 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.224+0000 7f01ad68aec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:30:02.282 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.295+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:02.320 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.327+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:30:02.326 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.340+0000 7f01ad68aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:30:02.406 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.420+0000 7f01ad68aec0 20 remove_watcher() i=0 2019-11-25T09:30:02.406 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.420+0000 7f01ad68aec0 2 removed watcher, disabling cache 2019-11-25T09:30:02.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.464+0000 7f01ad68aec0 20 remove_watcher() i=1 2019-11-25T09:30:02.497 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.511+0000 7f01ad68aec0 20 remove_watcher() i=2 2019-11-25T09:30:02.560 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.574+0000 7f01ad68aec0 20 remove_watcher() i=3 2019-11-25T09:30:02.620 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.634+0000 7f01ad68aec0 20 remove_watcher() i=4 2019-11-25T09:30:02.662 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.676+0000 7f01ad68aec0 20 remove_watcher() i=5 2019-11-25T09:30:02.707 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.721+0000 7f01ad68aec0 20 remove_watcher() i=6 2019-11-25T09:30:02.766 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:30:02.779+0000 7f01ad68aec0 20 remove_watcher() i=7 2019-11-25T09:30:02.777 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests AWS4Test 2019-11-25T09:30:03.428 INFO:teuthology.orchestra.run.smithi153.stdout:Starting a Gradle Daemon (subsequent builds will be faster) 2019-11-25T09:30:06.627 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.pom 2019-11-25T09:30:06.727 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/spring/gradle/dependency-management-plugin/1.0.3.RELEASE/dependency-management-plugin-1.0.3.RELEASE.jar 2019-11-25T09:30:07.829 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :clean UP-TO-DATE 2019-11-25T09:30:07.927 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-bom/1.11.549/aws-java-sdk-bom-1.11.549.pom 2019-11-25T09:30:07.927 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.549/aws-java-sdk-pom-1.11.549.pom 2019-11-25T09:30:08.128 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.pom 2019-11-25T09:30:08.128 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.pom 2019-11-25T09:30:08.128 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.pom 2019-11-25T09:30:08.128 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.pom 2019-11-25T09:30:08.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.pom 2019-11-25T09:30:08.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.pom 2019-11-25T09:30:08.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.pom 2019-11-25T09:30:08.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/3/oss-parent-3.pom 2019-11-25T09:30:08.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-parent/2.44.0/selenium-parent-2.44.0.pom 2019-11-25T09:30:08.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.1.3/commons-logging-1.1.3.pom 2019-11-25T09:30:08.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.pom 2019-11-25T09:30:08.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.pom 2019-11-25T09:30:08.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.pom 2019-11-25T09:30:08.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.pom 2019-11-25T09:30:08.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.pom 2019-11-25T09:30:08.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.2/jackson-parent-2.6.2.pom 2019-11-25T09:30:08.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.5.5/httpcomponents-client-4.5.5.pom 2019-11-25T09:30:08.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/28/commons-parent-28.pom 2019-11-25T09:30:08.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/24/oss-parent-24.pom 2019-11-25T09:30:08.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/10/httpcomponents-parent-10.pom 2019-11-25T09:30:08.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/18/apache-18.pom 2019-11-25T09:30:08.229 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/13/apache-13.pom 2019-11-25T09:30:08.328 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.pom 2019-11-25T09:30:08.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.pom 2019-11-25T09:30:08.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.pom 2019-11-25T09:30:08.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.pom 2019-11-25T09:30:08.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.pom 2019-11-25T09:30:08.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.pom 2019-11-25T09:30:08.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.pom 2019-11-25T09:30:08.330 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.pom 2019-11-25T09:30:08.330 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.pom 2019-11-25T09:30:08.332 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.pom 2019-11-25T09:30:08.332 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.pom 2019-11-25T09:30:08.333 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.pom 2019-11-25T09:30:08.333 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.pom 2019-11-25T09:30:08.333 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.pom 2019-11-25T09:30:08.333 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.pom 2019-11-25T09:30:08.333 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.pom 2019-11-25T09:30:08.334 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.pom 2019-11-25T09:30:08.334 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.pom 2019-11-25T09:30:08.334 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.pom 2019-11-25T09:30:08.334 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.pom 2019-11-25T09:30:08.429 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.pom 2019-11-25T09:30:08.430 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.pom 2019-11-25T09:30:08.430 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.pom 2019-11-25T09:30:08.430 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.pom 2019-11-25T09:30:08.430 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.pom 2019-11-25T09:30:08.431 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.pom 2019-11-25T09:30:08.431 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.pom 2019-11-25T09:30:08.431 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.pom 2019-11-25T09:30:08.431 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.pom 2019-11-25T09:30:08.431 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.pom 2019-11-25T09:30:08.431 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.pom 2019-11-25T09:30:08.431 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.pom 2019-11-25T09:30:08.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.pom 2019-11-25T09:30:08.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.pom 2019-11-25T09:30:08.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.pom 2019-11-25T09:30:08.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.pom 2019-11-25T09:30:08.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.pom 2019-11-25T09:30:08.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.pom 2019-11-25T09:30:08.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.pom 2019-11-25T09:30:08.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.pom 2019-11-25T09:30:08.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.pom 2019-11-25T09:30:08.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.pom 2019-11-25T09:30:08.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.pom 2019-11-25T09:30:08.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.pom 2019-11-25T09:30:08.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.pom 2019-11-25T09:30:08.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.pom 2019-11-25T09:30:08.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.pom 2019-11-25T09:30:08.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.pom 2019-11-25T09:30:08.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.pom 2019-11-25T09:30:08.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.pom 2019-11-25T09:30:08.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.pom 2019-11-25T09:30:08.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.pom 2019-11-25T09:30:08.435 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.pom 2019-11-25T09:30:08.435 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.pom 2019-11-25T09:30:08.435 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.pom 2019-11-25T09:30:08.435 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.pom 2019-11-25T09:30:08.528 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.pom 2019-11-25T09:30:08.529 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.pom 2019-11-25T09:30:08.529 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.pom 2019-11-25T09:30:08.529 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.pom 2019-11-25T09:30:08.529 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.pom 2019-11-25T09:30:08.529 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.pom 2019-11-25T09:30:08.529 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.pom 2019-11-25T09:30:08.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.pom 2019-11-25T09:30:08.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.pom 2019-11-25T09:30:08.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.pom 2019-11-25T09:30:08.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.pom 2019-11-25T09:30:08.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.pom 2019-11-25T09:30:08.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.pom 2019-11-25T09:30:08.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.pom 2019-11-25T09:30:08.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.pom 2019-11-25T09:30:08.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.pom 2019-11-25T09:30:08.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.pom 2019-11-25T09:30:08.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.pom 2019-11-25T09:30:08.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.pom 2019-11-25T09:30:08.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.pom 2019-11-25T09:30:08.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.pom 2019-11-25T09:30:08.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.pom 2019-11-25T09:30:08.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.pom 2019-11-25T09:30:08.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.pom 2019-11-25T09:30:08.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.pom 2019-11-25T09:30:08.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.pom 2019-11-25T09:30:08.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.pom 2019-11-25T09:30:08.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.pom 2019-11-25T09:30:08.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.pom 2019-11-25T09:30:08.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.pom 2019-11-25T09:30:08.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.pom 2019-11-25T09:30:08.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.pom 2019-11-25T09:30:08.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.pom 2019-11-25T09:30:08.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.pom 2019-11-25T09:30:08.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.pom 2019-11-25T09:30:08.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.pom 2019-11-25T09:30:08.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.pom 2019-11-25T09:30:08.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.pom 2019-11-25T09:30:08.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.pom 2019-11-25T09:30:08.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.pom 2019-11-25T09:30:08.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.pom 2019-11-25T09:30:08.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.pom 2019-11-25T09:30:08.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.pom 2019-11-25T09:30:08.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.pom 2019-11-25T09:30:08.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.pom 2019-11-25T09:30:08.633 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.pom 2019-11-25T09:30:08.633 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.pom 2019-11-25T09:30:08.633 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.pom 2019-11-25T09:30:08.633 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.pom 2019-11-25T09:30:08.633 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.pom 2019-11-25T09:30:08.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.pom 2019-11-25T09:30:08.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.pom 2019-11-25T09:30:08.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.pom 2019-11-25T09:30:08.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.pom 2019-11-25T09:30:08.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.pom 2019-11-25T09:30:08.635 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.pom 2019-11-25T09:30:08.641 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.pom 2019-11-25T09:30:08.641 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.pom 2019-11-25T09:30:08.642 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.pom 2019-11-25T09:30:08.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.pom 2019-11-25T09:30:08.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.pom 2019-11-25T09:30:08.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.pom 2019-11-25T09:30:08.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.pom 2019-11-25T09:30:08.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.pom 2019-11-25T09:30:08.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.pom 2019-11-25T09:30:08.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.pom 2019-11-25T09:30:08.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.pom 2019-11-25T09:30:08.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.pom 2019-11-25T09:30:08.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.pom 2019-11-25T09:30:08.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.pom 2019-11-25T09:30:08.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.pom 2019-11-25T09:30:08.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.pom 2019-11-25T09:30:08.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.pom 2019-11-25T09:30:08.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.pom 2019-11-25T09:30:08.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.pom 2019-11-25T09:30:08.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.pom 2019-11-25T09:30:08.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.pom 2019-11-25T09:30:08.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.pom 2019-11-25T09:30:08.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.pom 2019-11-25T09:30:08.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.pom 2019-11-25T09:30:08.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.pom 2019-11-25T09:30:08.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.pom 2019-11-25T09:30:08.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.pom 2019-11-25T09:30:08.732 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.pom 2019-11-25T09:30:08.732 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.pom 2019-11-25T09:30:08.732 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.pom 2019-11-25T09:30:08.732 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.pom 2019-11-25T09:30:08.732 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.pom 2019-11-25T09:30:08.732 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.pom 2019-11-25T09:30:08.732 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.pom 2019-11-25T09:30:08.733 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.pom 2019-11-25T09:30:08.733 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.pom 2019-11-25T09:30:08.733 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.pom 2019-11-25T09:30:08.733 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.pom 2019-11-25T09:30:08.832 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.pom 2019-11-25T09:30:08.833 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.pom 2019-11-25T09:30:08.833 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.pom 2019-11-25T09:30:08.833 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.pom 2019-11-25T09:30:08.833 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.pom 2019-11-25T09:30:08.833 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.pom 2019-11-25T09:30:08.833 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.pom 2019-11-25T09:30:08.834 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.pom 2019-11-25T09:30:08.834 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.pom 2019-11-25T09:30:08.834 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.pom 2019-11-25T09:30:08.834 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.pom 2019-11-25T09:30:08.834 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.pom 2019-11-25T09:30:08.834 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.pom 2019-11-25T09:30:08.835 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.pom 2019-11-25T09:30:08.835 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.pom 2019-11-25T09:30:08.835 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pom/1.11.22/aws-java-sdk-pom-1.11.22.pom 2019-11-25T09:30:08.835 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.pom 2019-11-25T09:30:08.835 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.pom 2019-11-25T09:30:08.835 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.pom 2019-11-25T09:30:08.835 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.pom 2019-11-25T09:30:08.836 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.pom 2019-11-25T09:30:08.836 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.pom 2019-11-25T09:30:08.836 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.pom 2019-11-25T09:30:08.836 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.pom 2019-11-25T09:30:08.836 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.pom 2019-11-25T09:30:08.837 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/project/6.1.9/project-6.1.9.pom 2019-11-25T09:30:08.837 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.9/commons-codec-1.9.pom 2019-11-25T09:30:08.837 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.pom 2019-11-25T09:30:08.935 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/32/commons-parent-32.pom 2019-11-25T09:30:08.935 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.pom 2019-11-25T09:30:08.935 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.pom 2019-11-25T09:30:08.935 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.pom 2019-11-25T09:30:08.936 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/beanshell/2.0b4/beanshell-2.0b4.pom 2019-11-25T09:30:08.936 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.pom 2019-11-25T09:30:08.936 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/34/commons-parent-34.pom 2019-11-25T09:30:08.936 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.pom 2019-11-25T09:30:08.936 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/35/commons-parent-35.pom 2019-11-25T09:30:09.028 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/15/apache-15.pom 2019-11-25T09:30:09.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.pom 2019-11-25T09:30:09.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-core/4.4.9/httpcomponents-core-4.4.9.pom 2019-11-25T09:30:09.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-parent/9/httpcomponents-parent-9.pom 2019-11-25T09:30:09.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.pom 2019-11-25T09:30:09.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.pom 2019-11-25T09:30:09.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/jackson-parent/2.6.1/jackson-parent-2.6.1.pom 2019-11-25T09:30:09.030 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/oss-parent/23/oss-parent-23.pom 2019-11-25T09:30:09.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.pom 2019-11-25T09:30:09.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.pom 2019-11-25T09:30:09.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-parent/4.1.17.Final/netty-parent-4.1.17.Final.pom 2019-11-25T09:30:09.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/9/oss-parent-9.pom 2019-11-25T09:30:09.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.pom 2019-11-25T09:30:09.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.pom 2019-11-25T09:30:09.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.pom 2019-11-25T09:30:09.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.pom 2019-11-25T09:30:09.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.pom 2019-11-25T09:30:09.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.pom 2019-11-25T09:30:09.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.pom 2019-11-25T09:30:09.131 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/6/oss-parent-6.pom 2019-11-25T09:30:09.131 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.pom 2019-11-25T09:30:09.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.pom 2019-11-25T09:30:09.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.pom 2019-11-25T09:30:09.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.pom 2019-11-25T09:30:09.228 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.pom 2019-11-25T09:30:09.229 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.pom 2019-11-25T09:30:09.229 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.pom 2019-11-25T09:30:09.229 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/25/commons-parent-25.pom 2019-11-25T09:30:09.229 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/17/commons-parent-17.pom 2019-11-25T09:30:09.328 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/7/apache-7.pom 2019-11-25T09:30:09.427 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/9/apache-9.pom 2019-11-25T09:30:09.427 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.pom 2019-11-25T09:30:09.427 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.pom 2019-11-25T09:30:09.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.pom 2019-11-25T09:30:09.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/sonatype/oss/oss-parent/7/oss-parent-7.pom 2019-11-25T09:30:09.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.pom 2019-11-25T09:30:09.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.pom 2019-11-25T09:30:09.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.pom 2019-11-25T09:30:09.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.pom 2019-11-25T09:30:09.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.pom 2019-11-25T09:30:09.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.pom 2019-11-25T09:30:09.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava-parent/18.0/guava-parent-18.0.pom 2019-11-25T09:30:09.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.pom 2019-11-25T09:30:09.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.pom 2019-11-25T09:30:09.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.pom 2019-11-25T09:30:09.534 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.pom 2019-11-25T09:30:09.534 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.pom 2019-11-25T09:30:09.627 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.pom 2019-11-25T09:30:09.627 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.pom 2019-11-25T09:30:09.628 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.pom 2019-11-25T09:30:09.628 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/9/commons-parent-9.pom 2019-11-25T09:30:09.628 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.pom 2019-11-25T09:30:09.628 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-parent/33/commons-parent-33.pom 2019-11-25T09:30:09.628 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-project/8.1.15.v20140411/jetty-project-8.1.15.v20140411.pom 2019-11-25T09:30:09.628 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcomponents-client/4.3.3/httpcomponents-client-4.3.3.pom 2019-11-25T09:30:09.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/apache/4/apache-4.pom 2019-11-25T09:30:09.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/project/7/project-7.pom 2019-11-25T09:30:09.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-parent/20/jetty-parent-20.pom 2019-11-25T09:30:09.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.pom 2019-11-25T09:30:09.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.pom 2019-11-25T09:30:09.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.pom 2019-11-25T09:30:09.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.pom 2019-11-25T09:30:09.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.pom 2019-11-25T09:30:09.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.pom 2019-11-25T09:30:10.428 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk/1.11.549/aws-java-sdk-1.11.549.jar 2019-11-25T09:30:10.428 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glacier/1.11.549/aws-java-sdk-glacier-1.11.549.jar 2019-11-25T09:30:10.428 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3control/1.11.549/aws-java-sdk-s3control-1.11.549.jar 2019-11-25T09:30:10.428 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sns/1.11.549/aws-java-sdk-sns-1.11.549.jar 2019-11-25T09:30:10.428 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-s3/1.11.549/aws-java-sdk-s3-1.11.549.jar 2019-11-25T09:30:10.429 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dynamodb/1.11.549/aws-java-sdk-dynamodb-1.11.549.jar 2019-11-25T09:30:10.429 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sqs/1.11.549/aws-java-sdk-sqs-1.11.549.jar 2019-11-25T09:30:10.429 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatchmetrics/1.11.549/aws-java-sdk-cloudwatchmetrics-1.11.549.jar 2019-11-25T09:30:10.429 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-managedblockchain/1.11.549/aws-java-sdk-managedblockchain-1.11.549.jar 2019-11-25T09:30:10.429 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-textract/1.11.549/aws-java-sdk-textract-1.11.549.jar 2019-11-25T09:30:10.429 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-docdb/1.11.549/aws-java-sdk-docdb-1.11.549.jar 2019-11-25T09:30:10.429 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-worklink/1.11.549/aws-java-sdk-worklink-1.11.549.jar 2019-11-25T09:30:10.430 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-backup/1.11.549/aws-java-sdk-backup-1.11.549.jar 2019-11-25T09:30:10.430 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewayv2/1.11.549/aws-java-sdk-apigatewayv2-1.11.549.jar 2019-11-25T09:30:10.430 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-apigatewaymanagementapi/1.11.549/aws-java-sdk-apigatewaymanagementapi-1.11.549.jar 2019-11-25T09:30:10.430 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kafka/1.11.549/aws-java-sdk-kafka-1.11.549.jar 2019-11-25T09:30:10.529 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-licensemanager/1.11.549/aws-java-sdk-licensemanager-1.11.549.jar 2019-11-25T09:30:10.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-securityhub/1.11.549/aws-java-sdk-securityhub-1.11.549.jar 2019-11-25T09:30:10.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appmesh/1.11.549/aws-java-sdk-appmesh-1.11.549.jar 2019-11-25T09:30:10.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fsx/1.11.549/aws-java-sdk-fsx-1.11.549.jar 2019-11-25T09:30:10.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconnect/1.11.549/aws-java-sdk-mediaconnect-1.11.549.jar 2019-11-25T09:30:10.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehendmedical/1.11.549/aws-java-sdk-comprehendmedical-1.11.549.jar 2019-11-25T09:30:10.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisanalyticsv2/1.11.549/aws-java-sdk-kinesisanalyticsv2-1.11.549.jar 2019-11-25T09:30:10.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datasync/1.11.549/aws-java-sdk-datasync-1.11.549.jar 2019-11-25T09:30:10.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-globalaccelerator/1.11.549/aws-java-sdk-globalaccelerator-1.11.549.jar 2019-11-25T09:30:10.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transfer/1.11.549/aws-java-sdk-transfer-1.11.549.jar 2019-11-25T09:30:10.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-robomaker/1.11.549/aws-java-sdk-robomaker-1.11.549.jar 2019-11-25T09:30:10.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-amplify/1.11.549/aws-java-sdk-amplify-1.11.549.jar 2019-11-25T09:30:10.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-quicksight/1.11.549/aws-java-sdk-quicksight-1.11.549.jar 2019-11-25T09:30:10.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rdsdata/1.11.549/aws-java-sdk-rdsdata-1.11.549.jar 2019-11-25T09:30:10.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53resolver/1.11.549/aws-java-sdk-route53resolver-1.11.549.jar 2019-11-25T09:30:10.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ram/1.11.549/aws-java-sdk-ram-1.11.549.jar 2019-11-25T09:30:10.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointsmsvoice/1.11.549/aws-java-sdk-pinpointsmsvoice-1.11.549.jar 2019-11-25T09:30:10.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-chime/1.11.549/aws-java-sdk-chime-1.11.549.jar 2019-11-25T09:30:10.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dlm/1.11.549/aws-java-sdk-dlm-1.11.549.jar 2019-11-25T09:30:10.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-eks/1.11.549/aws-java-sdk-eks-1.11.549.jar 2019-11-25T09:30:10.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpointemail/1.11.549/aws-java-sdk-pinpointemail-1.11.549.jar 2019-11-25T09:30:10.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-signer/1.11.549/aws-java-sdk-signer-1.11.549.jar 2019-11-25T09:30:10.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediatailor/1.11.549/aws-java-sdk-mediatailor-1.11.549.jar 2019-11-25T09:30:10.534 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pi/1.11.549/aws-java-sdk-pi-1.11.549.jar 2019-11-25T09:30:10.633 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-macie/1.11.549/aws-java-sdk-macie-1.11.549.jar 2019-11-25T09:30:10.633 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-neptune/1.11.549/aws-java-sdk-neptune-1.11.549.jar 2019-11-25T09:30:10.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickprojects/1.11.549/aws-java-sdk-iot1clickprojects-1.11.549.jar 2019-11-25T09:30:10.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot1clickdevices/1.11.549/aws-java-sdk-iot1clickdevices-1.11.549.jar 2019-11-25T09:30:10.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotanalytics/1.11.549/aws-java-sdk-iotanalytics-1.11.549.jar 2019-11-25T09:30:10.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-secretsmanager/1.11.549/aws-java-sdk-secretsmanager-1.11.549.jar 2019-11-25T09:30:10.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acmpca/1.11.549/aws-java-sdk-acmpca-1.11.549.jar 2019-11-25T09:30:10.634 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-connect/1.11.549/aws-java-sdk-connect-1.11.549.jar 2019-11-25T09:30:10.635 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-transcribe/1.11.549/aws-java-sdk-transcribe-1.11.549.jar 2019-11-25T09:30:10.635 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-fms/1.11.549/aws-java-sdk-fms-1.11.549.jar 2019-11-25T09:30:10.635 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscalingplans/1.11.549/aws-java-sdk-autoscalingplans-1.11.549.jar 2019-11-25T09:30:10.635 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workmail/1.11.549/aws-java-sdk-workmail-1.11.549.jar 2019-11-25T09:30:10.635 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicediscovery/1.11.549/aws-java-sdk-servicediscovery-1.11.549.jar 2019-11-25T09:30:10.636 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-serverlessapplicationrepository/1.11.549/aws-java-sdk-serverlessapplicationrepository-1.11.549.jar 2019-11-25T09:30:10.636 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloud9/1.11.549/aws-java-sdk-cloud9-1.11.549.jar 2019-11-25T09:30:10.636 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-alexaforbusiness/1.11.549/aws-java-sdk-alexaforbusiness-1.11.549.jar 2019-11-25T09:30:10.636 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-comprehend/1.11.549/aws-java-sdk-comprehend-1.11.549.jar 2019-11-25T09:30:10.636 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroups/1.11.549/aws-java-sdk-resourcegroups-1.11.549.jar 2019-11-25T09:30:10.636 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-translate/1.11.549/aws-java-sdk-translate-1.11.549.jar 2019-11-25T09:30:10.637 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemaker/1.11.549/aws-java-sdk-sagemaker-1.11.549.jar 2019-11-25T09:30:10.637 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sagemakerruntime/1.11.549/aws-java-sdk-sagemakerruntime-1.11.549.jar 2019-11-25T09:30:10.637 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appsync/1.11.549/aws-java-sdk-appsync-1.11.549.jar 2019-11-25T09:30:10.637 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iotjobsdataplane/1.11.549/aws-java-sdk-iotjobsdataplane-1.11.549.jar 2019-11-25T09:30:10.727 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-guardduty/1.11.549/aws-java-sdk-guardduty-1.11.549.jar 2019-11-25T09:30:10.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mq/1.11.549/aws-java-sdk-mq-1.11.549.jar 2019-11-25T09:30:10.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesisvideo/1.11.549/aws-java-sdk-kinesisvideo-1.11.549.jar 2019-11-25T09:30:10.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediaconvert/1.11.549/aws-java-sdk-mediaconvert-1.11.549.jar 2019-11-25T09:30:10.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastore/1.11.549/aws-java-sdk-mediastore-1.11.549.jar 2019-11-25T09:30:10.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediastoredata/1.11.549/aws-java-sdk-mediastoredata-1.11.549.jar 2019-11-25T09:30:10.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-medialive/1.11.549/aws-java-sdk-medialive-1.11.549.jar 2019-11-25T09:30:10.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mediapackage/1.11.549/aws-java-sdk-mediapackage-1.11.549.jar 2019-11-25T09:30:10.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costexplorer/1.11.549/aws-java-sdk-costexplorer-1.11.549.jar 2019-11-25T09:30:10.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pricing/1.11.549/aws-java-sdk-pricing-1.11.549.jar 2019-11-25T09:30:10.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mobile/1.11.549/aws-java-sdk-mobile-1.11.549.jar 2019-11-25T09:30:10.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsmv2/1.11.549/aws-java-sdk-cloudhsmv2-1.11.549.jar 2019-11-25T09:30:10.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-migrationhub/1.11.549/aws-java-sdk-migrationhub-1.11.549.jar 2019-11-25T09:30:10.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-glue/1.11.549/aws-java-sdk-glue-1.11.549.jar 2019-11-25T09:30:10.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplaceentitlement/1.11.549/aws-java-sdk-marketplaceentitlement-1.11.549.jar 2019-11-25T09:30:10.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codestar/1.11.549/aws-java-sdk-codestar-1.11.549.jar 2019-11-25T09:30:10.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-greengrass/1.11.549/aws-java-sdk-greengrass-1.11.549.jar 2019-11-25T09:30:10.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dax/1.11.549/aws-java-sdk-dax-1.11.549.jar 2019-11-25T09:30:10.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-athena/1.11.549/aws-java-sdk-athena-1.11.549.jar 2019-11-25T09:30:10.827 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-pinpoint/1.11.549/aws-java-sdk-pinpoint-1.11.549.jar 2019-11-25T09:30:10.827 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lexmodelbuilding/1.11.549/aws-java-sdk-lexmodelbuilding-1.11.549.jar 2019-11-25T09:30:10.827 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-resourcegroupstaggingapi/1.11.549/aws-java-sdk-resourcegroupstaggingapi-1.11.549.jar 2019-11-25T09:30:10.828 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworkscm/1.11.549/aws-java-sdk-opsworkscm-1.11.549.jar 2019-11-25T09:30:10.828 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-xray/1.11.549/aws-java-sdk-xray-1.11.549.jar 2019-11-25T09:30:10.828 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-support/1.11.549/aws-java-sdk-support-1.11.549.jar 2019-11-25T09:30:10.828 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpledb/1.11.549/aws-java-sdk-simpledb-1.11.549.jar 2019-11-25T09:30:10.828 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servermigration/1.11.549/aws-java-sdk-servermigration-1.11.549.jar 2019-11-25T09:30:10.828 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-servicecatalog/1.11.549/aws-java-sdk-servicecatalog-1.11.549.jar 2019-11-25T09:30:10.828 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-swf-libraries/1.11.22/aws-java-sdk-swf-libraries-1.11.22.jar 2019-11-25T09:30:10.829 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-route53/1.11.549/aws-java-sdk-route53-1.11.549.jar 2019-11-25T09:30:10.829 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-storagegateway/1.11.549/aws-java-sdk-storagegateway-1.11.549.jar 2019-11-25T09:30:10.829 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-simpleworkflow/1.11.549/aws-java-sdk-simpleworkflow-1.11.549.jar 2019-11-25T09:30:10.927 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-importexport/1.11.549/aws-java-sdk-importexport-1.11.549.jar 2019-11-25T09:30:10.927 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-sts/1.11.549/aws-java-sdk-sts-1.11.549.jar 2019-11-25T09:30:10.929 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rds/1.11.549/aws-java-sdk-rds-1.11.549.jar 2019-11-25T09:30:10.929 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-redshift/1.11.549/aws-java-sdk-redshift-1.11.549.jar 2019-11-25T09:30:10.929 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticbeanstalk/1.11.549/aws-java-sdk-elasticbeanstalk-1.11.549.jar 2019-11-25T09:30:10.929 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iam/1.11.549/aws-java-sdk-iam-1.11.549.jar 2019-11-25T09:30:10.929 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-datapipeline/1.11.549/aws-java-sdk-datapipeline-1.11.549.jar 2019-11-25T09:30:10.930 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancing/1.11.549/aws-java-sdk-elasticloadbalancing-1.11.549.jar 2019-11-25T09:30:11.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticache/1.11.549/aws-java-sdk-elasticache-1.11.549.jar 2019-11-25T09:30:11.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticloadbalancingv2/1.11.549/aws-java-sdk-elasticloadbalancingv2-1.11.549.jar 2019-11-25T09:30:11.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elastictranscoder/1.11.549/aws-java-sdk-elastictranscoder-1.11.549.jar 2019-11-25T09:30:11.029 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-budgets/1.11.549/aws-java-sdk-budgets-1.11.549.jar 2019-11-25T09:30:11.030 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-emr/1.11.549/aws-java-sdk-emr-1.11.549.jar 2019-11-25T09:30:11.030 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ec2/1.11.549/aws-java-sdk-ec2-1.11.549.jar 2019-11-25T09:30:11.030 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudwatch/1.11.549/aws-java-sdk-cloudwatch-1.11.549.jar 2019-11-25T09:30:11.030 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-logs/1.11.549/aws-java-sdk-logs-1.11.549.jar 2019-11-25T09:30:11.030 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidentity/1.11.549/aws-java-sdk-cognitoidentity-1.11.549.jar 2019-11-25T09:30:11.031 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-events/1.11.549/aws-java-sdk-events-1.11.549.jar 2019-11-25T09:30:11.031 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directconnect/1.11.549/aws-java-sdk-directconnect-1.11.549.jar 2019-11-25T09:30:11.031 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudtrail/1.11.549/aws-java-sdk-cloudtrail-1.11.549.jar 2019-11-25T09:30:11.031 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitosync/1.11.549/aws-java-sdk-cognitosync-1.11.549.jar 2019-11-25T09:30:11.031 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudformation/1.11.549/aws-java-sdk-cloudformation-1.11.549.jar 2019-11-25T09:30:11.128 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudfront/1.11.549/aws-java-sdk-cloudfront-1.11.549.jar 2019-11-25T09:30:11.128 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-clouddirectory/1.11.549/aws-java-sdk-clouddirectory-1.11.549.jar 2019-11-25T09:30:11.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-opsworks/1.11.549/aws-java-sdk-opsworks-1.11.549.jar 2019-11-25T09:30:11.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kinesis/1.11.549/aws-java-sdk-kinesis-1.11.549.jar 2019-11-25T09:30:11.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ses/1.11.549/aws-java-sdk-ses-1.11.549.jar 2019-11-25T09:30:11.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-autoscaling/1.11.549/aws-java-sdk-autoscaling-1.11.549.jar 2019-11-25T09:30:11.129 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudsearch/1.11.549/aws-java-sdk-cloudsearch-1.11.549.jar 2019-11-25T09:30:11.130 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codedeploy/1.11.549/aws-java-sdk-codedeploy-1.11.549.jar 2019-11-25T09:30:11.229 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codepipeline/1.11.549/aws-java-sdk-codepipeline-1.11.549.jar 2019-11-25T09:30:11.230 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-config/1.11.549/aws-java-sdk-config-1.11.549.jar 2019-11-25T09:30:11.230 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lambda/1.11.549/aws-java-sdk-lambda-1.11.549.jar 2019-11-25T09:30:11.230 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-kms/1.11.549/aws-java-sdk-kms-1.11.549.jar 2019-11-25T09:30:11.230 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecs/1.11.549/aws-java-sdk-ecs-1.11.549.jar 2019-11-25T09:30:11.327 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ecr/1.11.549/aws-java-sdk-ecr-1.11.549.jar 2019-11-25T09:30:11.327 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-ssm/1.11.549/aws-java-sdk-ssm-1.11.549.jar 2019-11-25T09:30:11.327 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workspaces/1.11.549/aws-java-sdk-workspaces-1.11.549.jar 2019-11-25T09:30:11.327 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cloudhsm/1.11.549/aws-java-sdk-cloudhsm-1.11.549.jar 2019-11-25T09:30:11.328 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-machinelearning/1.11.549/aws-java-sdk-machinelearning-1.11.549.jar 2019-11-25T09:30:11.328 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-directory/1.11.549/aws-java-sdk-directory-1.11.549.jar 2019-11-25T09:30:11.328 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-efs/1.11.549/aws-java-sdk-efs-1.11.549.jar 2019-11-25T09:30:11.328 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-devicefarm/1.11.549/aws-java-sdk-devicefarm-1.11.549.jar 2019-11-25T09:30:11.328 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacecommerceanalytics/1.11.549/aws-java-sdk-marketplacecommerceanalytics-1.11.549.jar 2019-11-25T09:30:11.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-inspector/1.11.549/aws-java-sdk-inspector-1.11.549.jar 2019-11-25T09:30:11.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-elasticsearch/1.11.549/aws-java-sdk-elasticsearch-1.11.549.jar 2019-11-25T09:30:11.329 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-waf/1.11.549/aws-java-sdk-waf-1.11.549.jar 2019-11-25T09:30:11.431 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-iot/1.11.549/aws-java-sdk-iot-1.11.549.jar 2019-11-25T09:30:11.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codecommit/1.11.549/aws-java-sdk-codecommit-1.11.549.jar 2019-11-25T09:30:11.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-acm/1.11.549/aws-java-sdk-acm-1.11.549.jar 2019-11-25T09:30:11.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-api-gateway/1.11.549/aws-java-sdk-api-gateway-1.11.549.jar 2019-11-25T09:30:11.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-gamelift/1.11.549/aws-java-sdk-gamelift-1.11.549.jar 2019-11-25T09:30:11.432 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-dms/1.11.549/aws-java-sdk-dms-1.11.549.jar 2019-11-25T09:30:11.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-marketplacemeteringservice/1.11.549/aws-java-sdk-marketplacemeteringservice-1.11.549.jar 2019-11-25T09:30:11.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-cognitoidp/1.11.549/aws-java-sdk-cognitoidp-1.11.549.jar 2019-11-25T09:30:11.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-discovery/1.11.549/aws-java-sdk-discovery-1.11.549.jar 2019-11-25T09:30:11.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-rekognition/1.11.549/aws-java-sdk-rekognition-1.11.549.jar 2019-11-25T09:30:11.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-applicationautoscaling/1.11.549/aws-java-sdk-applicationautoscaling-1.11.549.jar 2019-11-25T09:30:11.433 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-snowball/1.11.549/aws-java-sdk-snowball-1.11.549.jar 2019-11-25T09:30:11.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-polly/1.11.549/aws-java-sdk-polly-1.11.549.jar 2019-11-25T09:30:11.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-stepfunctions/1.11.549/aws-java-sdk-stepfunctions-1.11.549.jar 2019-11-25T09:30:11.434 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lightsail/1.11.549/aws-java-sdk-lightsail-1.11.549.jar 2019-11-25T09:30:11.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-health/1.11.549/aws-java-sdk-health-1.11.549.jar 2019-11-25T09:30:11.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-costandusagereport/1.11.549/aws-java-sdk-costandusagereport-1.11.549.jar 2019-11-25T09:30:11.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-codebuild/1.11.549/aws-java-sdk-codebuild-1.11.549.jar 2019-11-25T09:30:11.530 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-shield/1.11.549/aws-java-sdk-shield-1.11.549.jar 2019-11-25T09:30:11.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-appstream/1.11.549/aws-java-sdk-appstream-1.11.549.jar 2019-11-25T09:30:11.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-lex/1.11.549/aws-java-sdk-lex-1.11.549.jar 2019-11-25T09:30:11.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-batch/1.11.549/aws-java-sdk-batch-1.11.549.jar 2019-11-25T09:30:11.531 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-mechanicalturkrequester/1.11.549/aws-java-sdk-mechanicalturkrequester-1.11.549.jar 2019-11-25T09:30:11.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-organizations/1.11.549/aws-java-sdk-organizations-1.11.549.jar 2019-11-25T09:30:11.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-workdocs/1.11.549/aws-java-sdk-workdocs-1.11.549.jar 2019-11-25T09:30:11.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/testng/testng/6.1.1/testng-6.1.1.jar 2019-11-25T09:30:11.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-core/1.11.549/aws-java-sdk-core-1.11.549.jar 2019-11-25T09:30:11.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-htmlunit-driver/2.44.0/selenium-htmlunit-driver-2.44.0.jar 2019-11-25T09:30:11.532 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/log4j/log4j/1.2.17/log4j-1.2.17.jar 2019-11-25T09:30:11.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-java/2.44.0/selenium-java-2.44.0.jar 2019-11-25T09:30:11.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-server/2.44.0/selenium-server-2.44.0.jar 2019-11-25T09:30:11.533 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-chrome-driver/2.44.0/selenium-chrome-driver-2.44.0.jar 2019-11-25T09:30:11.628 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-firefox-driver/2.44.0/selenium-firefox-driver-2.44.0.jar 2019-11-25T09:30:11.628 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-ie-driver/2.44.0/selenium-ie-driver-2.44.0.jar 2019-11-25T09:30:11.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-safari-driver/2.44.0/selenium-safari-driver-2.44.0.jar 2019-11-25T09:30:11.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-support/2.44.0/selenium-support-2.44.0.jar 2019-11-25T09:30:11.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-remote-driver/2.44.0/selenium-remote-driver-2.44.0.jar 2019-11-25T09:30:11.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit/2.15/htmlunit-2.15.jar 2019-11-25T09:30:11.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-logging/commons-logging/1.2/commons-logging-1.2.jar 2019-11-25T09:30:11.629 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/software/amazon/ion/ion-java/1.0.2/ion-java-1.0.2.jar 2019-11-25T09:30:11.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpclient/4.5.5/httpclient-4.5.5.jar 2019-11-25T09:30:11.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/jmespath-java/1.11.549/jmespath-java-1.11.549.jar 2019-11-25T09:30:11.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpmime/4.3.3/httpmime-4.3.3.jar 2019-11-25T09:30:11.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-databind/2.6.7.2/jackson-databind-2.6.7.2.jar 2019-11-25T09:30:11.630 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/joda-time/joda-time/2.8.1/joda-time-2.8.1.jar 2019-11-25T09:30:11.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/dataformat/jackson-dataformat-cbor/2.6.7/jackson-dataformat-cbor-2.6.7.jar 2019-11-25T09:30:11.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcpkix-jdk15on/1.48/bcpkix-jdk15on-1.48.jar 2019-11-25T09:30:11.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/bouncycastle/bcprov-jdk15on/1.48/bcprov-jdk15on-1.48.jar 2019-11-25T09:30:11.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/mx4j/mx4j-tools/3.0.1/mx4j-tools-3.0.1.jar 2019-11-25T09:30:11.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/mortbay/jetty/servlet-api-2.5/6.1.9/servlet-api-2.5-6.1.9.jar 2019-11-25T09:30:11.631 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/amazonaws/aws-java-sdk-models/1.11.549/aws-java-sdk-models-1.11.549.jar 2019-11-25T09:30:11.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/jcip/jcip-annotations/1.0/jcip-annotations-1.0.jar 2019-11-25T09:30:11.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-repacked/7.6.1/jetty-repacked-7.6.1.jar 2019-11-25T09:30:11.632 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/jetty-rc-repacked/5/jetty-rc-repacked-5.jar 2019-11-25T09:30:11.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/yaml/snakeyaml/1.8/snakeyaml-1.8.jar 2019-11-25T09:30:11.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-codec/commons-codec/1.10/commons-codec-1.10.jar 2019-11-25T09:30:11.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/beanshell/bsh/2.0b4/bsh-2.0b4.jar 2019-11-25T09:30:11.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/junit/junit/3.8.1/junit-3.8.1.jar 2019-11-25T09:30:11.728 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/beust/jcommander/1.12/jcommander-1.12.jar 2019-11-25T09:30:11.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/httpcomponents/httpcore/4.4.9/httpcore-4.4.9.jar 2019-11-25T09:30:11.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-annotations/2.6.0/jackson-annotations-2.6.0.jar 2019-11-25T09:30:11.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/fasterxml/jackson/core/jackson-core/2.6.7/jackson-core-2.6.7.jar 2019-11-25T09:30:11.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec-http/4.1.17.Final/netty-codec-http-4.1.17.Final.jar 2019-11-25T09:30:11.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-handler/4.1.17.Final/netty-handler-4.1.17.Final.jar 2019-11-25T09:30:11.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/webbitserver/webbit/0.4.14/webbit-0.4.14.jar 2019-11-25T09:30:11.729 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-transport/4.1.17.Final/netty-transport-4.1.17.Final.jar 2019-11-25T09:30:11.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-io/commons-io/2.4/commons-io-2.4.jar 2019-11-25T09:30:11.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-exec/1.1/commons-exec-1.1.jar 2019-11-25T09:30:11.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-codec/4.1.17.Final/netty-codec-4.1.17.Final.jar 2019-11-25T09:30:11.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-buffer/4.1.17.Final/netty-buffer-4.1.17.Final.jar 2019-11-25T09:30:11.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/jna/3.4.0/jna-3.4.0.jar 2019-11-25T09:30:11.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/java/dev/jna/platform/3.4.0/platform-3.4.0.jar 2019-11-25T09:30:11.730 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty/3.5.2.Final/netty-3.5.2.Final.jar 2019-11-25T09:30:11.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-resolver/4.1.17.Final/netty-resolver-4.1.17.Final.jar 2019-11-25T09:30:11.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/io/netty/netty-common/4.1.17.Final/netty-common-4.1.17.Final.jar 2019-11-25T09:30:11.731 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/cglib/cglib-nodep/2.1_3/cglib-nodep-2.1_3.jar 2019-11-25T09:30:11.830 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/seleniumhq/selenium/selenium-api/2.44.0/selenium-api-2.44.0.jar 2019-11-25T09:30:11.830 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/google/code/gson/gson/2.3/gson-2.3.jar 2019-11-25T09:30:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/commons-collections/commons-collections/3.2.1/commons-collections-3.2.1.jar 2019-11-25T09:30:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/xalan/xalan/2.7.1/xalan-2.7.1.jar 2019-11-25T09:30:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/apache/commons/commons-lang3/3.3.2/commons-lang3-3.3.2.jar 2019-11-25T09:30:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/com/google/guava/guava/18.0/guava-18.0.jar 2019-11-25T09:30:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/xerces/xercesImpl/2.11.0/xercesImpl-2.11.0.jar 2019-11-25T09:30:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/nekohtml/nekohtml/1.9.21/nekohtml-1.9.21.jar 2019-11-25T09:30:11.831 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/cssparser/cssparser/0.9.14/cssparser-0.9.14.jar 2019-11-25T09:30:11.832 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-websocket/8.1.15.v20140411/jetty-websocket-8.1.15.v20140411.jar 2019-11-25T09:30:11.832 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/xalan/serializer/2.7.1/serializer-2.7.1.jar 2019-11-25T09:30:11.832 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/xml-apis/xml-apis/1.4.01/xml-apis-1.4.01.jar 2019-11-25T09:30:11.832 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/net/sourceforge/htmlunit/htmlunit-core-js/2.15/htmlunit-core-js-2.15.jar 2019-11-25T09:30:11.832 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/w3c/css/sac/1.3/sac-1.3.jar 2019-11-25T09:30:11.833 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-http/8.1.15.v20140411/jetty-http-8.1.15.v20140411.jar 2019-11-25T09:30:11.927 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-io/8.1.15.v20140411/jetty-io-8.1.15.v20140411.jar 2019-11-25T09:30:11.927 INFO:teuthology.orchestra.run.smithi153.stdout:Download https://repo.maven.apache.org/maven2/org/eclipse/jetty/jetty-util/8.1.15.v20140411/jetty-util-8.1.15.v20140411.jar 2019-11-25T09:30:21.026 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :compileJava 2019-11-25T09:30:21.127 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :processResources 2019-11-25T09:30:21.127 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :classes 2019-11-25T09:30:21.627 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:30:21.627 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :compileTestJava 2019-11-25T09:30:21.627 INFO:teuthology.orchestra.run.smithi153.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-11-25T09:30:21.628 INFO:teuthology.orchestra.run.smithi153.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-11-25T09:30:21.628 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:30:21.628 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :processTestResources NO-SOURCE 2019-11-25T09:30:21.628 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :testClasses 2019-11-25T09:30:29.426 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:30:29.426 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :test 2019-11-25T09:30:29.427 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:30:29.427 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 STARTED 2019-11-25T09:31:15.325 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:31:15.325 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadLLAPIAWS4 PASSED 2019-11-25T09:31:18.111 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:16:22.712 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:18.123 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:16:22.725 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:18.153 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:16:22.777 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:18.158 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:16:22.782 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:18.171 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:16:22.796 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:18.182 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:16:22.802 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:18.337 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:16:22.961 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:18.344 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:16:22.946 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:18.388 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:16:23.012 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:31:19.526 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:31:19.526 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 STARTED 2019-11-25T09:31:21.825 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:31:21.825 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testAbortMultipartUploadNotFoundLLAPIAWS4 PASSED 2019-11-25T09:31:28.925 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:31:28.925 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 STARTED 2019-11-25T09:32:05.325 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:32:05.325 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testDownloadHLAPIAWS4 PASSED 2019-11-25T09:32:14.124 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:32:14.125 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 STARTED 2019-11-25T09:32:14.224 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:32:14.224 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoBucketHLAPIAWS4 PASSED 2019-11-25T09:32:15.524 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:32:15.524 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 STARTED 2019-11-25T09:32:18.924 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:32:18.924 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testDownloadNoKeyHLAPIAWS4 PASSED 2019-11-25T09:32:23.924 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:32:23.924 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 STARTED 2019-11-25T09:32:24.148 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:17:28.751 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:24.177 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:17:28.779 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:24.190 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:17:28.815 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:24.193 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:17:28.818 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:24.208 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:17:28.833 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:24.216 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:17:28.837 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:24.239 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:17:28.864 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:24.248 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:17:28.851 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:24.250 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:17:28.875 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:32:49.424 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:32:49.424 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyHLAPIAWS4 PASSED 2019-11-25T09:33:02.424 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:33:02.424 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 STARTED 2019-11-25T09:36:28.218 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:21:32.818 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:36:28.284 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:21:32.910 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:36:28.288 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:21:32.913 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:36:28.292 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:21:32.892 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:36:28.335 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:21:32.961 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:36:28.337 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:21:32.962 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:36:28.446 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:21:33.072 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:36:28.473 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:21:33.098 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:36:28.482 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:21:33.082 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:37:20.227 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:37:20.227 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyMultipleSizesLLAPIAWS4 PASSED 2019-11-25T09:37:32.128 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:37:32.128 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 STARTED 2019-11-25T09:37:35.428 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:37:35.428 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCBucketHLAPIAWS4 PASSED 2019-11-25T09:37:40.728 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:37:40.729 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 STARTED 2019-11-25T09:37:44.429 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:37:44.429 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartCopyNoSRCKeyHLAPIAWS4 PASSED 2019-11-25T09:37:55.229 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:37:55.229 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 STARTED 2019-11-25T09:38:11.234 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:38:11.234 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadHLAPIAWS4 PASSED 2019-11-25T09:38:17.831 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:38:17.831 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 STARTED 2019-11-25T09:38:18.031 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:38:18.031 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadNoBucketHLAPIAWS4 PASSED 2019-11-25T09:38:18.431 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:38:18.431 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 STARTED 2019-11-25T09:38:39.632 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:38:39.632 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartDownloadWithPauseHLAPIAWS4 PASSED 2019-11-25T09:38:47.232 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:38:47.232 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 STARTED 2019-11-25T09:39:26.834 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:39:26.834 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadHLAPIAWS4 PASSED 2019-11-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:39:41.534 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 STARTED 2019-11-25T09:40:03.035 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:40:03.035 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadIncorrectMissingPartLLAPIAWS4 PASSED 2019-11-25T09:40:13.535 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:40:13.536 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 STARTED 2019-11-25T09:40:45.967 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:25:50.557 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:40:45.977 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:25:50.568 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:40:45.984 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:25:50.597 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:40:46.051 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:25:50.664 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:40:46.051 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:25:50.664 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:40:46.057 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:25:50.670 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:40:46.084 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:25:50.675 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:40:46.146 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:25:50.759 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:40:46.185 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:25:50.798 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.410 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:31:57.994 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.420 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:31:58.005 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.430 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:31:58.036 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.437 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:31:58.043 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.441 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:31:58.048 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.443 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:31:58.050 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.653 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:31:58.259 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.677 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:31:58.284 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:46:53.679 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:31:58.263 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:47:51.743 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:47:51.743 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadMultipleSizesLLAPIAWS4 PASSED 2019-11-25T09:47:57.243 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:47:57.243 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 STARTED 2019-11-25T09:48:41.043 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:48:41.043 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadOverwriteExistingObjectLLAPIAWS4 PASSED 2019-11-25T09:48:48.743 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:48:48.744 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 STARTED 2019-11-25T09:49:29.844 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:49:29.844 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadSmallLLAPIAWS4 PASSED 2019-11-25T09:49:38.944 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:49:38.944 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 STARTED 2019-11-25T09:49:46.844 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:49:46.844 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testMultipartUploadWithPauseAWS4 PASSED 2019-11-25T09:49:59.744 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:49:59.744 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 STARTED 2019-11-25T09:50:02.544 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:02.544 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateAfterToday4AWS4 PASSED 2019-11-25T09:50:10.344 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:10.344 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 STARTED 2019-11-25T09:50:12.844 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:12.844 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeEpochAWS4 PASSED 2019-11-25T09:50:19.044 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:19.044 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 STARTED 2019-11-25T09:50:22.144 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:22.144 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateBeforeToday4AWS4 PASSED 2019-11-25T09:50:29.644 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:29.644 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 STARTED 2019-11-25T09:50:31.844 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:31.844 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateEmptyAWS4 PASSED 2019-11-25T09:50:40.444 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:40.444 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 STARTED 2019-11-25T09:50:44.044 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:44.044 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateInvalidAWS4 PASSED 2019-11-25T09:50:52.444 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:52.444 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 STARTED 2019-11-25T09:50:56.044 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:50:56.044 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadDateNoneAWS4 PASSED 2019-11-25T09:51:03.444 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:51:03.444 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 STARTED 2019-11-25T09:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:51:05.744 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testObjectCreateBadMd5InvalidGarbageAWS4 PASSED 2019-11-25T09:51:12.144 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:51:12.144 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 STARTED 2019-11-25T09:51:43.344 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:51:43.344 INFO:teuthology.orchestra.run.smithi153.stdout:suite > AWS4 tests > AWS4Test.testUploadFileHLAPIBigFileAWS4 PASSED 2019-11-25T09:51:50.744 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:51:50.744 INFO:teuthology.orchestra.run.smithi153.stdout:---------------------------------------------------------------------- 2019-11-25T09:51:50.744 INFO:teuthology.orchestra.run.smithi153.stdout:| Results: SUCCESS (26 tests, 26 successes, 0 failures, 0 skipped) | 2019-11-25T09:51:50.744 INFO:teuthology.orchestra.run.smithi153.stdout:---------------------------------------------------------------------- 2019-11-25T09:51:50.829 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:51:50.829 INFO:teuthology.orchestra.run.smithi153.stdout:BUILD SUCCESSFUL in 21m 47s 2019-11-25T09:51:50.829 INFO:teuthology.orchestra.run.smithi153.stdout:5 actionable tasks: 4 executed, 1 up-to-date 2019-11-25T09:51:51.148 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:51:51.338 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.335+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.374+0000 7f65ce0d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:51:51.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.374+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.386 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.383+0000 7f65ce0d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:51:51.386 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.383+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.395 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.392+0000 7f65ce0d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:51:51.395 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.392+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.416 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.413+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:51:51.655 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.652+0000 7f65ce0d6ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:51:51.655 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.652+0000 7f65ce0d6ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:51:51.655 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.652+0000 7f65ce0d6ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:51:51.655 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.652+0000 7f65ce0d6ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:51:51.656 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.652+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.663+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:51:51.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.663+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.677 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.674+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:51:51.677 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.674+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.690 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.686+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:51:51.690 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.686+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.702 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.698+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:51:51.702 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.698+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.709 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.705+0000 7f65ce0d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:51:51.709 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.705+0000 7f65ce0d6ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:51:51.709 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.705+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.711+0000 7f65ce0d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:51:51.714 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.711+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.725+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:51:51.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.725+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.738 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.735+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:51:51.738 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.735+0000 7f65ce0d6ec0 20 zonegroup default 2019-11-25T09:51:51.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.735+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.744 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.741+0000 7f65ce0d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:51:51.744 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.741+0000 7f65ce0d6ec0 10 Cannot find current period zone using local zone 2019-11-25T09:51:51.744 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.741+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.747+0000 7f65ce0d6ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:51:51.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.747+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.756+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:51:51.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.756+0000 7f65ce0d6ec0 20 rados->read ofs=0 len=0 2019-11-25T09:51:51.771 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.768+0000 7f65ce0d6ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:51:51.771 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.768+0000 7f65ce0d6ec0 20 zone default found 2019-11-25T09:51:51.771 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.768+0000 7f65ce0d6ec0 20 started sync module instance, tier type = 2019-11-25T09:51:51.771 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:51.768+0000 7f65ce0d6ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:51:52.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.172+0000 7f65ce0d6ec0 20 add_watcher() i=0 2019-11-25T09:51:52.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.172+0000 7f65ce0d6ec0 20 add_watcher() i=1 2019-11-25T09:51:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.172+0000 7f65ce0d6ec0 20 add_watcher() i=2 2019-11-25T09:51:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.172+0000 7f65ce0d6ec0 20 add_watcher() i=3 2019-11-25T09:51:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.172+0000 7f65ce0d6ec0 20 add_watcher() i=4 2019-11-25T09:51:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.172+0000 7f65ce0d6ec0 20 add_watcher() i=5 2019-11-25T09:51:52.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.172+0000 7f65ce0d6ec0 20 add_watcher() i=6 2019-11-25T09:51:52.209 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.206+0000 7f65ce0d6ec0 20 add_watcher() i=7 2019-11-25T09:51:52.209 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.206+0000 7f65ce0d6ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:51:52.209 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.206+0000 7f658bfdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:51:52.222 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.218+0000 7f65bb304700 0 -- 172.21.15.153:0/789135494 >> v2:172.21.15.81:3300/0 conn(0x55a27f972a70 msgr2=0x55a27f971af0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:51:52.260 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:52.256+0000 7f65bbb05700 0 -- 172.21.15.153:0/4005758976 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55a27f95da00 msgr2=0x55a27f96b770 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:51:53.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.662+0000 7f64fa7fc700 20 reqs_thread_entry: start 2019-11-25T09:51:53.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.662+0000 7f64f3fff700 20 reqs_thread_entry: start 2019-11-25T09:51:53.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.662+0000 7f65ce0d6ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:51:53.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.662+0000 7f64f27fc700 20 reqs_thread_entry: start 2019-11-25T09:51:53.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.662+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:51:53.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.805+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:53.853 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.850+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:51:53.889 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.417', time=2019-11-25T11:32:57.418331+0000, chain.objs.size()=9 2019-11-25T09:51:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.1 2019-11-25T09:51:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__shadow_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.1_1 2019-11-25T09:51:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.2 2019-11-25T09:51:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__shadow_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.2_1 2019-11-25T09:51:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.3 2019-11-25T09:51:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__shadow_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.3_1 2019-11-25T09:51:53.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.4 2019-11-25T09:51:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__shadow_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.4_1 2019-11-25T09:51:53.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:53.885+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.5 2019-11-25T09:51:54.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.154+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:51:54.241 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.238+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:51:54.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.281+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:54.310 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.307+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:54.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.319+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:51:54.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.319+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.430', time=2019-11-25T11:37:26.932860+0000, chain.objs.size()=5 2019-11-25T09:51:54.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.319+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.19__shadow_.AZ2F5RknTaaM6RWFQ6XH8Btc0TwSFjf_1 2019-11-25T09:51:54.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.319+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.19__shadow_.AZ2F5RknTaaM6RWFQ6XH8Btc0TwSFjf_2 2019-11-25T09:51:54.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.319+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.19__shadow_.AZ2F5RknTaaM6RWFQ6XH8Btc0TwSFjf_3 2019-11-25T09:51:54.325 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.319+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.19__shadow_.AZ2F5RknTaaM6RWFQ6XH8Btc0TwSFjf_4 2019-11-25T09:51:54.325 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.319+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.19__shadow_.AZ2F5RknTaaM6RWFQ6XH8Btc0TwSFjf_5 2019-11-25T09:51:54.512 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.508+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:51:54.626 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.623+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:51:54.672 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.669+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:54.687 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.684+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:54.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.692+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:54.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.722+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:51:54.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.764+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:54.783 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.780+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:54.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.810+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-11-25T09:51:54.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.810+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.559', time=2019-11-25T11:39:29.984211+0000, chain.objs.size()=9 2019-11-25T09:51:54.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.810+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.1 2019-11-25T09:51:54.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.810+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__shadow_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.1_1 2019-11-25T09:51:54.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.2 2019-11-25T09:51:54.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__shadow_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.2_1 2019-11-25T09:51:54.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.3 2019-11-25T09:51:54.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__shadow_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.3_1 2019-11-25T09:51:54.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.4 2019-11-25T09:51:54.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__shadow_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.4_1 2019-11-25T09:51:54.816 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file.mpg.2~Mw3xvyLdhjMZEuD5AO_DBoxPxx7om5O.5 2019-11-25T09:51:54.816 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.560', time=2019-11-25T11:39:33.000720+0000, chain.objs.size()=9 2019-11-25T09:51:54.816 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.1 2019-11-25T09:51:54.816 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__shadow_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.1_1 2019-11-25T09:51:54.816 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:54.811+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.2 2019-11-25T09:51:55.071 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.067+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__shadow_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.2_1 2019-11-25T09:51:55.071 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.067+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.3 2019-11-25T09:51:55.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.154+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__shadow_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.3_1 2019-11-25T09:51:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.4 2019-11-25T09:51:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__shadow_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.4_1 2019-11-25T09:51:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.26__multipart_file2.mpg.2~HX7O3LNrH6foRNo7FTMgAy2OG_ctu-9.5 2019-11-25T09:51:55.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.608', time=2019-11-25T11:44:37.880469+0000, chain.objs.size()=21 2019-11-25T09:51:55.196 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.1 2019-11-25T09:51:55.196 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.1_1 2019-11-25T09:51:55.196 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.2 2019-11-25T09:51:55.196 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.2_1 2019-11-25T09:51:55.197 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.192+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.3 2019-11-25T09:51:55.271 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.268+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.3_1 2019-11-25T09:51:55.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.346+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.4 2019-11-25T09:51:55.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.481+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.4_1 2019-11-25T09:51:55.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.5 2019-11-25T09:51:55.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.5_1 2019-11-25T09:51:55.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.6 2019-11-25T09:51:55.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.6_1 2019-11-25T09:51:55.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.7 2019-11-25T09:51:55.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.7_1 2019-11-25T09:51:55.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.8 2019-11-25T09:51:55.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.8_1 2019-11-25T09:51:55.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.9 2019-11-25T09:51:55.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.541+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.9_1 2019-11-25T09:51:55.680 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.677+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.10 2019-11-25T09:51:55.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.775+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.10_1 2019-11-25T09:51:55.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.775+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~12nWR2qhd58I8EQjTquIA7NiJKwDPLF.11 2019-11-25T09:51:55.994 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:55.987+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:51:56.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.154+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:51:56.229 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.226+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:56.252 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.246+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:51:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.460', time=2019-11-25T11:36:41.048909+0000, chain.objs.size()=7 2019-11-25T09:51:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~U74oN2aE8_UW9I8D94rlY4R3RGWJItC.1 2019-11-25T09:51:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~U74oN2aE8_UW9I8D94rlY4R3RGWJItC.1_1 2019-11-25T09:51:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~U74oN2aE8_UW9I8D94rlY4R3RGWJItC.1_2 2019-11-25T09:51:56.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~U74oN2aE8_UW9I8D94rlY4R3RGWJItC.2 2019-11-25T09:51:56.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~U74oN2aE8_UW9I8D94rlY4R3RGWJItC.2_1 2019-11-25T09:51:56.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~U74oN2aE8_UW9I8D94rlY4R3RGWJItC.2_2 2019-11-25T09:51:56.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~U74oN2aE8_UW9I8D94rlY4R3RGWJItC.3 2019-11-25T09:51:56.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.836', time=2019-11-25T11:51:46.315876+0000, chain.objs.size()=21 2019-11-25T09:51:56.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.1 2019-11-25T09:51:56.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.1_1 2019-11-25T09:51:56.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.2 2019-11-25T09:51:56.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.2_1 2019-11-25T09:51:56.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.3 2019-11-25T09:51:56.512 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.509+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.3_1 2019-11-25T09:51:56.605 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.602+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.4 2019-11-25T09:51:56.605 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.602+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.4_1 2019-11-25T09:51:56.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.5 2019-11-25T09:51:56.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.5_1 2019-11-25T09:51:56.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.6 2019-11-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.6_1 2019-11-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.7 2019-11-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.7_1 2019-11-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.8 2019-11-25T09:51:56.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.8_1 2019-11-25T09:51:56.721 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.718+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.9 2019-11-25T09:51:56.842 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.839+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.9_1 2019-11-25T09:51:56.842 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:56.839+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.10 2019-11-25T09:51:57.035 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.032+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__shadow_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.10_1 2019-11-25T09:51:57.036 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.032+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.39__multipart_key1.2~u7L6V7Vom7XJIPA8yYKghK64PFyTa5A.11 2019-11-25T09:51:57.203 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.199+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:51:57.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.284+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:51:57.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.329+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:57.346 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.343+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:57.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:51:57.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.415', time=2019-11-25T11:32:50.972097+0000, chain.objs.size()=9 2019-11-25T09:51:57.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.1 2019-11-25T09:51:57.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__shadow_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.1_1 2019-11-25T09:51:57.363 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.2 2019-11-25T09:51:57.363 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__shadow_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.2_1 2019-11-25T09:51:57.363 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.3 2019-11-25T09:51:57.363 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__shadow_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.3_1 2019-11-25T09:51:57.363 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.4 2019-11-25T09:51:57.363 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.358+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__shadow_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.4_1 2019-11-25T09:51:57.363 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.359+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.17__multipart_key1.2~-mzIp2CsKlWjETdiVA472kRCMUML8zW.5 2019-11-25T09:51:57.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.359+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.472', time=2019-11-25T11:37:22.234043+0000, chain.objs.size()=7 2019-11-25T09:51:57.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.359+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~KQjHnwCplOP9g8BPgcvyDltfYainmny.1 2019-11-25T09:51:57.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.359+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~KQjHnwCplOP9g8BPgcvyDltfYainmny.1_1 2019-11-25T09:51:57.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.359+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~KQjHnwCplOP9g8BPgcvyDltfYainmny.1_2 2019-11-25T09:51:57.538 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.535+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~KQjHnwCplOP9g8BPgcvyDltfYainmny.2 2019-11-25T09:51:57.635 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.632+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~KQjHnwCplOP9g8BPgcvyDltfYainmny.2_1 2019-11-25T09:51:57.687 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.684+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~KQjHnwCplOP9g8BPgcvyDltfYainmny.2_2 2019-11-25T09:51:57.688 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.684+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~KQjHnwCplOP9g8BPgcvyDltfYainmny.3 2019-11-25T09:51:57.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:57.952+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:51:58.057 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.054+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:51:58.130 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.127+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:58.146 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.142+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:58.152 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.149+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:58.193 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.190+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:51:58.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.258+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:58.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.281+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:58.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:51:58.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.386', time=2019-11-25T11:32:07.388438+0000, chain.objs.size()=9 2019-11-25T09:51:58.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__multipart_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.1 2019-11-25T09:51:58.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__shadow_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.1_1 2019-11-25T09:51:58.302 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__multipart_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.2 2019-11-25T09:51:58.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__shadow_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.2_1 2019-11-25T09:51:58.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__multipart_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.3 2019-11-25T09:51:58.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__shadow_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.3_1 2019-11-25T09:51:58.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__multipart_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.4 2019-11-25T09:51:58.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__shadow_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.4_1 2019-11-25T09:51:58.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.298+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.15__multipart_key1.2~7wHOSe204GZgNl1xpav7IlU4MtIZWEI.5 2019-11-25T09:51:58.599 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.596+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:51:58.689 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.686+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:51:58.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.736+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:58.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.756+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:58.767 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.764+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:58.807 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.804+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:51:58.875 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.871+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:51:58.898 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.895+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:51:58.927 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:51:58.927 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~WyKdum68QORLPsjvkcIoheCJT_nDBg7', time=2019-11-25T11:49:56.081121+0000, chain.objs.size()=39 2019-11-25T09:51:58.927 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.1 2019-11-25T09:51:58.927 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.2 2019-11-25T09:51:58.927 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.3 2019-11-25T09:51:58.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.4 2019-11-25T09:51:58.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.5 2019-11-25T09:51:58.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.6 2019-11-25T09:51:58.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.7 2019-11-25T09:51:58.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.8 2019-11-25T09:51:58.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.9 2019-11-25T09:51:58.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.10 2019-11-25T09:51:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.11 2019-11-25T09:51:58.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:58.923+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.12 2019-11-25T09:51:59.028 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.025+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.13 2019-11-25T09:51:59.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.133+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.14 2019-11-25T09:51:59.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.133+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.15 2019-11-25T09:51:59.192 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.189+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.16 2019-11-25T09:51:59.192 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.189+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.17 2019-11-25T09:51:59.203 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.200+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.18 2019-11-25T09:51:59.272 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.269+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.k1n5IONTMX0YTWyB_gjbmxPAvIfx6qZ.19 2019-11-25T09:51:59.272 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.269+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.h2SEigs2wlLxq3xoB2U_-CxiN6tWfWF.20 2019-11-25T09:51:59.272 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.269+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.D5PKeo2kzqVRPxFfWgeh0OowQFJ82Ne.21 2019-11-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.269+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.22 2019-11-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.269+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.23 2019-11-25T09:51:59.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.269+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.24 2019-11-25T09:51:59.315 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.312+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.25 2019-11-25T09:51:59.393 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.384+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.26 2019-11-25T09:51:59.518 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.514+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.27 2019-11-25T09:51:59.518 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.514+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.28 2019-11-25T09:51:59.518 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.514+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.29 2019-11-25T09:51:59.518 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.514+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.30 2019-11-25T09:51:59.518 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.514+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.31 2019-11-25T09:51:59.821 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.818+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.32 2019-11-25T09:51:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.818+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.33 2019-11-25T09:51:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.818+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.34 2019-11-25T09:51:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.818+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.35 2019-11-25T09:51:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.818+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.36 2019-11-25T09:51:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.818+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.37 2019-11-25T09:51:59.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.818+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.38 2019-11-25T09:51:59.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:51:59.818+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.31__multipart_key1.2~WyKdum68QORLPsjvkcIoheCJT_nDBg7.39 2019-11-25T09:52:00.222 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.219+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:00.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.321+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:52:00.373 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.369+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:00.394 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.391+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:00.424 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:00.424 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.628', time=2019-11-25T11:46:54.683982+0000, chain.objs.size()=16 2019-11-25T09:52:00.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.1 2019-11-25T09:52:00.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.1_1 2019-11-25T09:52:00.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.1_2 2019-11-25T09:52:00.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.2 2019-11-25T09:52:00.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.2_1 2019-11-25T09:52:00.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.2_2 2019-11-25T09:52:00.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.3 2019-11-25T09:52:00.426 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.3_1 2019-11-25T09:52:00.426 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.3_2 2019-11-25T09:52:00.426 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.4 2019-11-25T09:52:00.426 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.4_1 2019-11-25T09:52:00.426 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.420+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.4_2 2019-11-25T09:52:00.697 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.691+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.5 2019-11-25T09:52:00.697 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.691+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.5_1 2019-11-25T09:52:00.714 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.711+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.5_2 2019-11-25T09:52:00.714 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:00.711+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~Z-UJmsu6tiayWwAIgZgcvqnU3MktE5n.6 2019-11-25T09:52:01.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.009+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:01.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.094+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:52:01.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.148+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:01.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.171+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:01.183 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.180+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:01.218 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.215+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:52:01.295 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.292+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:01.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.315+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:01.338 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:01.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi', time=2019-11-25T11:31:15.075511+0000, chain.objs.size()=9 2019-11-25T09:52:01.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__multipart_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.1 2019-11-25T09:52:01.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__shadow_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.1_1 2019-11-25T09:52:01.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__multipart_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.2 2019-11-25T09:52:01.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__shadow_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.2_1 2019-11-25T09:52:01.341 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__multipart_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.3 2019-11-25T09:52:01.341 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__shadow_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.3_1 2019-11-25T09:52:01.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__multipart_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.4 2019-11-25T09:52:01.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__shadow_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.4_1 2019-11-25T09:52:01.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.334+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.13__multipart_key1.2~cesKJBT90SmC1ejhMRfza7SWuQ-ZZAi.5 2019-11-25T09:52:01.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.724+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:01.862 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.856+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:52:01.901 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.898+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:01.919 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.915+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:01.927 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.924+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:01.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:01.966+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:52:02.011 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.008+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:02.027 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.024+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:02.035 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.032+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:02.065 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.061+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:52:02.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.100+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:02.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.115+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:02.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.123+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:02.161 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.157+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:52:02.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.229+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:02.260 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.256+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:02.283 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.278+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:52:02.283 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.278+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.528', time=2019-11-25T11:38:41.482451+0000, chain.objs.size()=9 2019-11-25T09:52:02.283 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.278+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__multipart_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.1 2019-11-25T09:52:02.284 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.278+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__shadow_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.1_1 2019-11-25T09:52:02.284 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__multipart_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.2 2019-11-25T09:52:02.284 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__shadow_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.2_1 2019-11-25T09:52:02.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__multipart_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.3 2019-11-25T09:52:02.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__shadow_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.3_1 2019-11-25T09:52:02.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__multipart_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.4 2019-11-25T09:52:02.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__shadow_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.4_1 2019-11-25T09:52:02.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.25__multipart_key1.2~lQ__0d_sOfxbsOpixfcmTVMxK8gWeh_.5 2019-11-25T09:52:02.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.636', time=2019-11-25T11:47:50.691191+0000, chain.objs.size()=16 2019-11-25T09:52:02.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.1 2019-11-25T09:52:02.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.1_1 2019-11-25T09:52:02.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.279+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.1_2 2019-11-25T09:52:02.536 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.532+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.2 2019-11-25T09:52:02.589 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.586+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.2_1 2019-11-25T09:52:02.589 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.586+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.2_2 2019-11-25T09:52:02.589 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.586+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.3 2019-11-25T09:52:02.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.3_1 2019-11-25T09:52:02.660 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.3_2 2019-11-25T09:52:02.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.4 2019-11-25T09:52:02.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.657+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.4_1 2019-11-25T09:52:02.688 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.684+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.4_2 2019-11-25T09:52:02.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.710+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.5 2019-11-25T09:52:02.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.710+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.5_1 2019-11-25T09:52:02.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.812+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.5_2 2019-11-25T09:52:02.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:02.812+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~ACWkLxdUFXhqKrB57FDDXrmTjr5z5ke.6 2019-11-25T09:52:03.223 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.220+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:03.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.329+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:52:03.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.382+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:03.417 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.414+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:03.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:52:03.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.595', time=2019-11-25T11:43:23.210796+0000, chain.objs.size()=21 2019-11-25T09:52:03.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.1 2019-11-25T09:52:03.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.1_1 2019-11-25T09:52:03.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.2 2019-11-25T09:52:03.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.2_1 2019-11-25T09:52:03.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.3 2019-11-25T09:52:03.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.3_1 2019-11-25T09:52:03.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.4 2019-11-25T09:52:03.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.4_1 2019-11-25T09:52:03.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.5 2019-11-25T09:52:03.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.5_1 2019-11-25T09:52:03.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.6 2019-11-25T09:52:03.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.448+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.6_1 2019-11-25T09:52:03.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.740+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.7 2019-11-25T09:52:03.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:03.740+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.7_1 2019-11-25T09:52:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.8 2019-11-25T09:52:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.8_1 2019-11-25T09:52:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.9 2019-11-25T09:52:04.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.9_1 2019-11-25T09:52:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.10 2019-11-25T09:52:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.10_1 2019-11-25T09:52:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~AN3Jnfh9_Qs3obkfMaHych32exV93Y8.11 2019-11-25T09:52:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.644', time=2019-11-25T11:47:53.193617+0000, chain.objs.size()=16 2019-11-25T09:52:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.1 2019-11-25T09:52:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.1_1 2019-11-25T09:52:04.080 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.1_2 2019-11-25T09:52:04.080 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.074+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.2 2019-11-25T09:52:04.422 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.419+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.2_1 2019-11-25T09:52:04.423 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.419+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.2_2 2019-11-25T09:52:04.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.3 2019-11-25T09:52:04.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.3_1 2019-11-25T09:52:04.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.3_2 2019-11-25T09:52:04.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.4 2019-11-25T09:52:04.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.4_1 2019-11-25T09:52:04.577 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.4_2 2019-11-25T09:52:04.577 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.5 2019-11-25T09:52:04.577 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.5_1 2019-11-25T09:52:04.577 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.572+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.5_2 2019-11-25T09:52:04.590 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:37:09.173 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.600 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:37:09.183 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.605 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:37:09.211 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.607 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:37:09.213 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.616 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:37:09.221 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.618 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:37:09.223 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.796 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:37:09.378 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.822 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:37:09.428 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.825 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:37:09.431 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:52:04.933 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:04.930+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~mJUX9E3k7tvfUiJMVA19S63yG4LzYAZ.6 2019-11-25T09:52:05.122 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.119+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:05.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.377+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:52:05.447 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.444+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:05.491 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.487+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:05.507 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.504+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:05.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.564+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:52:05.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.638+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:05.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.680+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:05.697 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.694+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:05.697 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.694+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~FIfCca5bInx3sZ32u8YWL8sG3rPXoYl', time=2019-11-25T11:40:07.223064+0000, chain.objs.size()=5 2019-11-25T09:52:05.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.694+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.27__multipart_key1.2~FIfCca5bInx3sZ32u8YWL8sG3rPXoYl.7 2019-11-25T09:52:05.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.694+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.27__shadow_key1.2~FIfCca5bInx3sZ32u8YWL8sG3rPXoYl.7_1 2019-11-25T09:52:05.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.694+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.27__multipart_key1.2~FIfCca5bInx3sZ32u8YWL8sG3rPXoYl.10 2019-11-25T09:52:05.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.694+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.27__shadow_key1.2~FIfCca5bInx3sZ32u8YWL8sG3rPXoYl.10_1 2019-11-25T09:52:05.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.694+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.27__multipart_key1.2~FIfCca5bInx3sZ32u8YWL8sG3rPXoYl.13 2019-11-25T09:52:05.885 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.882+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:05.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:05.951+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:52:06.020 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.017+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:06.038 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.035+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:06.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.050+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:06.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.050+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.620', time=2019-11-25T11:45:41.626432+0000, chain.objs.size()=19 2019-11-25T09:52:06.055 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.050+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.1 2019-11-25T09:52:06.055 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.050+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.1_1 2019-11-25T09:52:06.055 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.2 2019-11-25T09:52:06.055 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.2_1 2019-11-25T09:52:06.055 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.3 2019-11-25T09:52:06.055 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.3_1 2019-11-25T09:52:06.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.4 2019-11-25T09:52:06.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.4_1 2019-11-25T09:52:06.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.5 2019-11-25T09:52:06.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.5_1 2019-11-25T09:52:06.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.6 2019-11-25T09:52:06.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.051+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.6_1 2019-11-25T09:52:06.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.481+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.7 2019-11-25T09:52:06.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.481+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.7_1 2019-11-25T09:52:06.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.481+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.8 2019-11-25T09:52:06.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.481+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.8_1 2019-11-25T09:52:06.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.481+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.9 2019-11-25T09:52:06.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.481+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__shadow_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.9_1 2019-11-25T09:52:06.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.481+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.28__multipart_key1.2~FQq6KpQohtdVipzf_MCXvqrRTqG9uh6.10 2019-11-25T09:52:06.853 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.849+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:06.942 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.939+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:52:06.980 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:06.977+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:07.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.010+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:07.021 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.018+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:07.058 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.055+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:52:07.108 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.105+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:07.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.124+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:07.136 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.133+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:07.183 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.180+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:52:07.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.223+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:07.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.250+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:07.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.270+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:07.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.270+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.438', time=2019-11-25T11:35:19.385223+0000, chain.objs.size()=9 2019-11-25T09:52:07.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.270+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.1 2019-11-25T09:52:07.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.1_1 2019-11-25T09:52:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.2 2019-11-25T09:52:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.2_1 2019-11-25T09:52:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.3 2019-11-25T09:52:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.3_1 2019-11-25T09:52:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.4 2019-11-25T09:52:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.4_1 2019-11-25T09:52:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.271+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~H0fx0Z4WtZdsEp9BHzAlAl1cDb28nrz.5 2019-11-25T09:52:07.556 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.553+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:07.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.755+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:52:07.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.819+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:07.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.848+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.865+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.865+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.446', time=2019-11-25T11:35:52.545646+0000, chain.objs.size()=9 2019-11-25T09:52:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.865+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.1 2019-11-25T09:52:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.865+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.1_1 2019-11-25T09:52:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.865+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.2 2019-11-25T09:52:07.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.865+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.2_1 2019-11-25T09:52:07.870 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.866+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.3 2019-11-25T09:52:07.870 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.866+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.3_1 2019-11-25T09:52:07.870 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.866+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.4 2019-11-25T09:52:07.870 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.866+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.4_1 2019-11-25T09:52:07.870 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:07.866+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~E2Im0I21iJotTjDzlsFgLgYt_cMnfhR.5 2019-11-25T09:52:08.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.254+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:08.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.339+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:52:08.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.377+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:08.395 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.392+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:08.412 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.408+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T09:52:08.412 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.408+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.509', time=2019-11-25T11:38:12.561771+0000, chain.objs.size()=9 2019-11-25T09:52:08.412 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.408+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__multipart_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.1 2019-11-25T09:52:08.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.408+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__shadow_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.1_1 2019-11-25T09:52:08.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__multipart_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.2 2019-11-25T09:52:08.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__shadow_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.2_1 2019-11-25T09:52:08.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__multipart_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.3 2019-11-25T09:52:08.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__shadow_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.3_1 2019-11-25T09:52:08.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__multipart_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.4 2019-11-25T09:52:08.414 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__shadow_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.4_1 2019-11-25T09:52:08.414 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.24__multipart_key1.2~5smBFUZRMcnuXDE9-mOydP3vCRERSDC.5 2019-11-25T09:52:08.414 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.674', time=2019-11-25T11:49:32.426602+0000, chain.objs.size()=9 2019-11-25T09:52:08.414 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__multipart_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.1 2019-11-25T09:52:08.414 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__shadow_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.1_1 2019-11-25T09:52:08.414 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.409+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__multipart_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.2 2019-11-25T09:52:08.777 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.774+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__shadow_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.2_1 2019-11-25T09:52:08.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.774+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__multipart_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.3 2019-11-25T09:52:08.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.774+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__shadow_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.3_1 2019-11-25T09:52:08.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.774+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__multipart_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.4 2019-11-25T09:52:08.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.774+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__shadow_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.4_1 2019-11-25T09:52:08.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:08.774+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.30__multipart_key1.2~Jnla_D6FMkHpsjcvNpfkrM58ulg8CpW.5 2019-11-25T09:52:09.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.115+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:09.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.269+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:52:09.330 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.326+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:09.348 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.345+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:09.359 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.355+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:09.390 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.386+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:52:09.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.421+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:09.440 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.436+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:09.448 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.444+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:09.490 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.486+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:52:09.536 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.533+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:09.556 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.553+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:09.569 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.566+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:09.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.596+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:52:09.676 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.673+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:09.705 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.701+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:09.726 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.722+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:09.726 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.722+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.659', time=2019-11-25T11:48:43.838672+0000, chain.objs.size()=9 2019-11-25T09:52:09.726 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.722+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__multipart_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.1 2019-11-25T09:52:09.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.722+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__shadow_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.1_1 2019-11-25T09:52:09.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.722+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__multipart_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.2 2019-11-25T09:52:09.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.722+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__shadow_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.2_1 2019-11-25T09:52:09.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.722+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__multipart_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.3 2019-11-25T09:52:09.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.722+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__shadow_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.3_1 2019-11-25T09:52:09.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.723+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__multipart_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.4 2019-11-25T09:52:09.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.723+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__shadow_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.4_1 2019-11-25T09:52:09.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:09.723+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.29__multipart_key1.2~3DOB6MrqsAgVcobvXv5MFIJFC0CkGjP.5 2019-11-25T09:52:10.024 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.021+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:10.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.185+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:52:10.229 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.226+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:10.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.244+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.454', time=2019-11-25T11:36:20.196135+0000, chain.objs.size()=9 2019-11-25T09:52:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.1 2019-11-25T09:52:10.261 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.1_1 2019-11-25T09:52:10.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.2 2019-11-25T09:52:10.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.2_1 2019-11-25T09:52:10.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.3 2019-11-25T09:52:10.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.3_1 2019-11-25T09:52:10.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.4 2019-11-25T09:52:10.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.4_1 2019-11-25T09:52:10.263 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.257+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~Q_Tl8GojbIPMehYitM6A3CUQROmKxLD.5 2019-11-25T09:52:10.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.605+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:10.734 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.730+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:52:10.776 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.773+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:10.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.788+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.804+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T09:52:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.804+0000 7f65ce0d6ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.466', time=2019-11-25T11:37:18.724097+0000, chain.objs.size()=7 2019-11-25T09:52:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.804+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~Mdycx0FOqg4DXy7POHurn8cWdtcK0Ya.1 2019-11-25T09:52:10.808 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.804+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~Mdycx0FOqg4DXy7POHurn8cWdtcK0Ya.1_1 2019-11-25T09:52:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.804+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~Mdycx0FOqg4DXy7POHurn8cWdtcK0Ya.1_2 2019-11-25T09:52:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.804+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~Mdycx0FOqg4DXy7POHurn8cWdtcK0Ya.2 2019-11-25T09:52:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.804+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~Mdycx0FOqg4DXy7POHurn8cWdtcK0Ya.2_1 2019-11-25T09:52:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.805+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__shadow_key1.2~Mdycx0FOqg4DXy7POHurn8cWdtcK0Ya.2_2 2019-11-25T09:52:10.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:10.805+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.20__multipart_key1.2~Mdycx0FOqg4DXy7POHurn8cWdtcK0Ya.3 2019-11-25T09:52:11.082 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.079+0000 7f65ce0d6ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T09:52:11.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.254+0000 7f65ce0d6ec0 20 remove_watcher() i=0 2019-11-25T09:52:11.258 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.254+0000 7f65ce0d6ec0 2 removed watcher, disabling cache 2019-11-25T09:52:11.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.297+0000 7f65ce0d6ec0 20 remove_watcher() i=1 2019-11-25T09:52:11.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.362+0000 7f65ce0d6ec0 20 remove_watcher() i=2 2019-11-25T09:52:11.407 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.404+0000 7f65ce0d6ec0 20 remove_watcher() i=3 2019-11-25T09:52:11.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.462+0000 7f65ce0d6ec0 20 remove_watcher() i=4 2019-11-25T09:52:11.500 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.497+0000 7f65ce0d6ec0 20 remove_watcher() i=5 2019-11-25T09:52:11.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.548+0000 7f65ce0d6ec0 20 remove_watcher() i=6 2019-11-25T09:52:11.602 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.599+0000 7f65ce0d6ec0 20 remove_watcher() i=7 2019-11-25T09:52:11.611 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:52:11.755 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.752+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:11.783 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.780+0000 7f23bf57aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:11.783 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.780+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:11.790 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.787+0000 7f23bf57aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:11.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.787+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:11.799 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.796+0000 7f23bf57aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:11.800 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.796+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:11.828 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:11.824+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:12.041 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.038+0000 7f23bf57aec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:52:12.041 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.038+0000 7f23bf57aec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:52:12.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.038+0000 7f23bf57aec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:52:12.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.038+0000 7f23bf57aec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:52:12.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.038+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.053+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:12.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.053+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.068 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.065+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:52:12.069 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.065+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.080 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.077+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:12.080 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.077+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.093 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.090+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:52:12.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.090+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.104 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.100+0000 7f23bf57aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:12.104 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.100+0000 7f23bf57aec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:52:12.104 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.100+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.110+0000 7f23bf57aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:12.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.110+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.133 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.130+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:12.133 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.130+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.147+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:52:12.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.147+0000 7f23bf57aec0 20 zonegroup default 2019-11-25T09:52:12.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.147+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.161 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.158+0000 7f23bf57aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:12.161 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.158+0000 7f23bf57aec0 10 Cannot find current period zone using local zone 2019-11-25T09:52:12.161 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.158+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.167+0000 7f23bf57aec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:12.171 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.167+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.190 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.186+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:12.190 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.186+0000 7f23bf57aec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:12.205 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.202+0000 7f23bf57aec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:52:12.205 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.202+0000 7f23bf57aec0 20 zone default found 2019-11-25T09:52:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.202+0000 7f23bf57aec0 20 started sync module instance, tier type = 2019-11-25T09:52:12.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.202+0000 7f23bf57aec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:52:12.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.563+0000 7f23bf57aec0 20 add_watcher() i=0 2019-11-25T09:52:12.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.563+0000 7f23bf57aec0 20 add_watcher() i=1 2019-11-25T09:52:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.563+0000 7f23bf57aec0 20 add_watcher() i=2 2019-11-25T09:52:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.563+0000 7f23bf57aec0 20 add_watcher() i=3 2019-11-25T09:52:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.563+0000 7f23bf57aec0 20 add_watcher() i=4 2019-11-25T09:52:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.563+0000 7f23bf57aec0 20 add_watcher() i=5 2019-11-25T09:52:12.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.563+0000 7f23bf57aec0 20 add_watcher() i=6 2019-11-25T09:52:12.596 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.593+0000 7f23bf57aec0 20 add_watcher() i=7 2019-11-25T09:52:12.597 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.593+0000 7f23bf57aec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:52:12.597 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.593+0000 7f237bfef700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:52:12.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.597+0000 7f23acfa9700 0 -- 172.21.15.153:0/1259321282 >> v2:172.21.15.153:3300/0 conn(0x558a2d02adf0 msgr2=0x558a2d02b380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:52:12.637 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:12.634+0000 7f23ac7a8700 0 -- 172.21.15.153:0/931970860 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x558a2d0292e0 msgr2=0x558a2d0625a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:52:14.351 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.347+0000 7f22ea7fc700 20 reqs_thread_entry: start 2019-11-25T09:52:14.351 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.348+0000 7f22e3fff700 20 reqs_thread_entry: start 2019-11-25T09:52:14.351 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.348+0000 7f23bf57aec0 20 init_complete bucket index max shards: 0 2019-11-25T09:52:14.352 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.349+0000 7f22e27fc700 20 reqs_thread_entry: start 2019-11-25T09:52:14.352 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.349+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:52:14.428 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.425+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:14.456 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.453+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:14.469 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.465+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:14.519 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.516+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:52:14.592 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.589+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:14.618 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.615+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:14.631 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.628+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:14.674 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.671+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:52:14.724 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.720+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:14.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.747+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:14.765 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.762+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:14.803 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.800+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:52:14.867 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.864+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:14.888 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.885+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:14.899 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.896+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:14.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:14.937+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:52:15.007 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.004+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.034+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:15.049 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.046+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.097 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.094+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:52:15.156 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.153+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.190+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:15.215 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.212+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.266+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:52:15.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.316+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.336+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:15.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.346+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.396 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.392+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:52:15.468 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.465+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.492 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.489+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:15.505 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.502+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.559 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.556+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:52:15.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.650+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.678 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.675+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:15.691 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.688+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.724+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:52:15.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.791+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.811+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:15.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.819+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.861 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.858+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:52:15.900 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.897+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.916 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.913+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:15.926 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.923+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:15.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:15.964+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:52:16.007 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.004+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.025 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.022+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:16.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.034+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.067+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:52:16.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.124+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.146+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:16.161 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.158+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.203+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:52:16.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.259+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.279 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.276+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:16.290 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.287+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.334 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.331+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:52:16.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.410+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.445 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.442+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:16.464 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.461+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.504+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:52:16.583 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.580+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.617 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.613+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:16.636 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.632+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.688 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.684+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:52:16.765 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.761+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.800 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.797+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:16.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.812+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.868 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.865+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:52:16.946 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.943+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:16.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.975+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:16.994 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:16.991+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.039+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:52:17.125 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.122+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.154 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.150+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:17.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.163+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.220+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:52:17.308 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.304+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.337+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:17.355 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.352+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.438+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:52:17.496 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.493+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.518 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.515+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:17.530 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.527+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.575+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:52:17.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.661+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.696 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.693+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:17.711 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.708+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.755 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.752+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:52:17.835 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.832+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.859 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.856+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:17.883 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.868+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:17.907 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.904+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:52:17.989 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:17.986+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.029 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.025+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:18.052 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.049+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.122 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.119+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:52:18.180 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.177+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.215 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.212+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:18.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.221+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.264 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.261+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:52:18.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.304+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.322 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.319+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:18.334 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.331+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.361+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:52:18.401 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.398+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.417 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.414+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:18.425 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.422+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.451 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.448+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:52:18.487 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.484+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.500+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:18.512 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.509+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.538 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.535+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:52:18.611 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.608+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.639 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.636+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:18.653 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.650+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.697 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.694+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:52:18.762 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.758+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.783 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.780+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:18.792 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.789+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.844 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.840+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:52:18.883 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.880+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.900 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.897+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:18.909 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.906+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:18.952 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:18.949+0000 7f23bf57aec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:52:19.021 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.018+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:19.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.039+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:19.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.050+0000 7f23bf57aec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:19.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.146+0000 7f23bf57aec0 20 remove_watcher() i=0 2019-11-25T09:52:19.149 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.146+0000 7f23bf57aec0 2 removed watcher, disabling cache 2019-11-25T09:52:19.202 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.199+0000 7f23bf57aec0 20 remove_watcher() i=1 2019-11-25T09:52:19.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.243+0000 7f23bf57aec0 20 remove_watcher() i=2 2019-11-25T09:52:19.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.300+0000 7f23bf57aec0 20 remove_watcher() i=3 2019-11-25T09:52:19.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.346+0000 7f23bf57aec0 20 remove_watcher() i=4 2019-11-25T09:52:19.383 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.380+0000 7f23bf57aec0 20 remove_watcher() i=5 2019-11-25T09:52:19.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.426+0000 7f23bf57aec0 20 remove_watcher() i=6 2019-11-25T09:52:19.459 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.456+0000 7f23bf57aec0 20 remove_watcher() i=7 2019-11-25T09:52:19.466 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:52:19.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.576+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.602 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.599+0000 7f5782f13ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:19.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.599+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.611 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.608+0000 7f5782f13ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:19.611 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.608+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.623 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.620+0000 7f5782f13ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:19.624 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.620+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.640 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.637+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:19.859 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.856+0000 7f5782f13ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:52:19.859 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.856+0000 7f5782f13ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:52:19.859 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.856+0000 7f5782f13ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:52:19.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.856+0000 7f5782f13ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:52:19.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.856+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.868 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.865+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:19.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.865+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.878 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.874+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:52:19.878 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.874+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.883+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:19.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.883+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.898 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.895+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:52:19.898 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.895+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.902 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.899+0000 7f5782f13ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:19.902 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.899+0000 7f5782f13ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:52:19.902 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.899+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.903+0000 7f5782f13ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:19.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.903+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.911+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:19.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.911+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.920+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:52:19.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.920+0000 7f5782f13ec0 20 zonegroup default 2019-11-25T09:52:19.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.920+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.925+0000 7f5782f13ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:19.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.925+0000 7f5782f13ec0 10 Cannot find current period zone using local zone 2019-11-25T09:52:19.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.925+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.932 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.929+0000 7f5782f13ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:19.933 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.929+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.938+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:19.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.938+0000 7f5782f13ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:19.949 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.946+0000 7f5782f13ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:52:19.949 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.946+0000 7f5782f13ec0 20 zone default found 2019-11-25T09:52:19.949 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.946+0000 7f5782f13ec0 20 started sync module instance, tier type = 2019-11-25T09:52:19.950 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:19.946+0000 7f5782f13ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:52:20.269 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.266+0000 7f5782f13ec0 20 add_watcher() i=0 2019-11-25T09:52:20.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.266+0000 7f5782f13ec0 20 add_watcher() i=1 2019-11-25T09:52:20.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.266+0000 7f5782f13ec0 20 add_watcher() i=2 2019-11-25T09:52:20.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.266+0000 7f5782f13ec0 20 add_watcher() i=3 2019-11-25T09:52:20.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.266+0000 7f5782f13ec0 20 add_watcher() i=4 2019-11-25T09:52:20.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.266+0000 7f5782f13ec0 20 add_watcher() i=5 2019-11-25T09:52:20.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.266+0000 7f5782f13ec0 20 add_watcher() i=6 2019-11-25T09:52:20.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.296+0000 7f5782f13ec0 20 add_watcher() i=7 2019-11-25T09:52:20.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.296+0000 7f5782f13ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:52:20.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.296+0000 7f5740fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:52:20.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.306+0000 7f5770942700 0 -- 172.21.15.153:0/177210784 >> v2:172.21.15.81:3300/0 conn(0x55dcb6067270 msgr2=0x55dcb6086eb0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:52:20.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:20.338+0000 7f5770141700 0 -- 172.21.15.153:0/15295478 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7f56b4017570 msgr2=0x7f56b4019a80 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:52:21.843 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.840+0000 7f56b2ffd700 20 reqs_thread_entry: start 2019-11-25T09:52:21.844 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.840+0000 7f56b0ff9700 20 reqs_thread_entry: start 2019-11-25T09:52:21.844 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.840+0000 7f5782f13ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:52:21.844 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.840+0000 7f56a6ffd700 20 reqs_thread_entry: start 2019-11-25T09:52:21.844 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.840+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:52:21.897 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.894+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:21.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.916+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:21.931 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.927+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:21.970 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:21.967+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:52:22.030 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.026+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.058 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.055+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:22.068 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.065+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.114+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:52:22.202 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.198+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.221+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:22.235 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.232+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.282 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.278+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:52:22.365 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.362+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.398+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:22.412 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.408+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.445 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.442+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:52:22.492 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.488+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.514 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.511+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:22.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.522+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.555 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.552+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:52:22.605 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.602+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.626+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:22.640 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.637+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.695+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:52:22.765 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.762+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.789 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.785+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:22.808 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.805+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.856 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.852+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:52:22.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.920+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:22.949 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.945+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:22.960 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:22.957+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.009 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.006+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:52:23.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.077+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.101 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.098+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:23.110 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.107+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.145 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.142+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:52:23.187 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.184+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.204 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.200+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:23.213 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.209+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.243+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:52:23.289 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.286+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.305 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.302+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:23.316 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.313+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.347 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.343+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:52:23.392 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.389+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.411 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.408+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:23.421 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.418+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.454 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.451+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:52:23.497 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.494+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.517 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.514+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:23.529 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.526+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.574 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.570+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:52:23.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.631+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.658+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:23.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.680+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.745+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:52:23.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.818+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.856 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.853+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:23.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.866+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.912 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.909+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:52:23.969 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.966+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:23.987 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.984+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:23.996 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:23.993+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.038 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.034+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:52:24.084 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.080+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.101 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.098+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:24.110 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.107+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.159 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.155+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:52:24.211 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.207+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.232 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.229+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:24.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.239+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.281 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.278+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:52:24.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.341+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.370 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.367+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:24.378 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.375+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.433+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:52:24.507 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.503+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.523+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:24.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.533+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.572 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.569+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:52:24.633 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.617+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.636 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.633+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:24.644 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.641+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.674+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:52:24.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.730+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.746+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:24.758 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.755+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:24.805 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.802+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:52:24.985 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:24.982+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.008 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.005+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:25.019 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.016+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.067 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.064+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:52:25.133 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.130+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.164+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:25.201 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.198+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.243+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:52:25.322 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.318+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.369 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.360+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:25.378 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.375+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.431 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.428+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:52:25.519 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.516+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.559 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.556+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:25.596 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.592+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.674 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.671+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:52:25.731 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.728+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.751 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.748+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:25.762 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.759+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.801 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.797+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:52:25.842 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.838+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.875 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.872+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:25.887 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.884+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:25.947 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.944+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:52:25.988 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:25.985+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:26.006 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.003+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:26.015 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.012+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:26.063 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.060+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:52:26.145 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.141+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:26.168 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.165+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:26.180 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.177+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:26.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.223+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:52:26.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.282+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:26.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.300+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:26.313 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.310+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:26.353 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.349+0000 7f5782f13ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:52:26.409 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.406+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:26.430 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.427+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:26.439 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.436+0000 7f5782f13ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:26.538 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.534+0000 7f5782f13ec0 20 remove_watcher() i=0 2019-11-25T09:52:26.538 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.534+0000 7f5782f13ec0 2 removed watcher, disabling cache 2019-11-25T09:52:26.578 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.575+0000 7f5782f13ec0 20 remove_watcher() i=1 2019-11-25T09:52:26.612 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.609+0000 7f5782f13ec0 20 remove_watcher() i=2 2019-11-25T09:52:26.657 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.654+0000 7f5782f13ec0 20 remove_watcher() i=3 2019-11-25T09:52:26.693 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.690+0000 7f5782f13ec0 20 remove_watcher() i=4 2019-11-25T09:52:26.724 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.721+0000 7f5782f13ec0 20 remove_watcher() i=5 2019-11-25T09:52:26.756 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.753+0000 7f5782f13ec0 20 remove_watcher() i=6 2019-11-25T09:52:26.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.783+0000 7f5782f13ec0 20 remove_watcher() i=7 2019-11-25T09:52:26.794 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:52:26.903 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.900+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:26.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.927+0000 7f5302796ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:26.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.927+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:26.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.937+0000 7f5302796ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:26.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.937+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:26.951 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.948+0000 7f5302796ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:26.951 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.948+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:26.986 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:26.983+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:27.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.222+0000 7f5302796ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:52:27.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.222+0000 7f5302796ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:52:27.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.222+0000 7f5302796ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:52:27.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.222+0000 7f5302796ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:52:27.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.222+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.241 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.238+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:27.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.238+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.253+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:52:27.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.253+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.272 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.269+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:27.273 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.269+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.290 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.287+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:52:27.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.287+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.297+0000 7f5302796ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:27.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.297+0000 7f5302796ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:52:27.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.297+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.314 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.311+0000 7f5302796ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:27.314 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.311+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.326+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:27.329 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.326+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.346 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.343+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:52:27.347 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.343+0000 7f5302796ec0 20 zonegroup default 2019-11-25T09:52:27.347 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.343+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.358 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.355+0000 7f5302796ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:27.358 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.355+0000 7f5302796ec0 10 Cannot find current period zone using local zone 2019-11-25T09:52:27.358 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.355+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.366 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.362+0000 7f5302796ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:52:27.366 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.362+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.383 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.380+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:52:27.384 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.380+0000 7f5302796ec0 20 rados->read ofs=0 len=0 2019-11-25T09:52:27.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.399+0000 7f5302796ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:52:27.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.399+0000 7f5302796ec0 20 zone default found 2019-11-25T09:52:27.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.399+0000 7f5302796ec0 20 started sync module instance, tier type = 2019-11-25T09:52:27.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.399+0000 7f5302796ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:52:27.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.812+0000 7f5302796ec0 20 add_watcher() i=0 2019-11-25T09:52:27.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.812+0000 7f5302796ec0 20 add_watcher() i=1 2019-11-25T09:52:27.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.812+0000 7f5302796ec0 20 add_watcher() i=2 2019-11-25T09:52:27.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.812+0000 7f5302796ec0 20 add_watcher() i=3 2019-11-25T09:52:27.816 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.812+0000 7f5302796ec0 20 add_watcher() i=4 2019-11-25T09:52:27.816 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.812+0000 7f5302796ec0 20 add_watcher() i=5 2019-11-25T09:52:27.816 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.812+0000 7f5302796ec0 20 add_watcher() i=6 2019-11-25T09:52:27.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.848+0000 7f5302796ec0 20 add_watcher() i=7 2019-11-25T09:52:27.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.848+0000 7f5302796ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:52:27.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.848+0000 7f5243fff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:52:27.853 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.850+0000 7f52f01c5700 0 -- 172.21.15.153:0/2720599891 >> v2:172.21.15.81:3300/0 conn(0x556e2accbe40 msgr2=0x556e2accb1d0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:52:27.894 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:27.891+0000 7f52ef9c4700 0 -- 172.21.15.153:0/1340787373 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7f5230017580 msgr2=0x7f5230019a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T09:52:29.426 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.423+0000 7f522e7fc700 20 reqs_thread_entry: start 2019-11-25T09:52:29.426 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.423+0000 7f5227fff700 20 reqs_thread_entry: start 2019-11-25T09:52:29.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.423+0000 7f5302796ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:52:29.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.424+0000 7f52267fc700 20 reqs_thread_entry: start 2019-11-25T09:52:29.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.424+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:52:29.496 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.493+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:29.524 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.521+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:29.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.542+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:29.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.597+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:52:29.685 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.681+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:29.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.710+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:29.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.725+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:29.780 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.777+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:52:29.837 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.833+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:29.854 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.851+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:29.861 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.858+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:29.904 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.901+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:52:29.998 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:29.995+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.029 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.026+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:30.041 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.038+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.098 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.095+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:52:30.165 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.162+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.189 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.186+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:30.201 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.198+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.248 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.245+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:52:30.305 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.302+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.331 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.328+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:30.347 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.344+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.390 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.387+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:52:30.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.449+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.473 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.470+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:30.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.480+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.538 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.535+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:52:30.596 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.593+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.613+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:30.626 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.623+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.668 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.665+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:52:30.722 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.718+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.740 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.737+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:30.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.747+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.782+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:52:30.839 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.836+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.861 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.857+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:30.870 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.867+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:30.909 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.906+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:52:30.985 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:30.982+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.038+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:31.053 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.050+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.097 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.093+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:52:31.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.164+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.207 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.191+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:31.208 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.203+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.262 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.257+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:52:31.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.334+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.357+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:31.373 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.369+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.426 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.422+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:52:31.492 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.487+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.514 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.510+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:31.527 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.522+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.571 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.567+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:52:31.632 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.628+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.656 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.651+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:31.678 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.673+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.720+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:52:31.783 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.779+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.803 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.798+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:31.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.808+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.856 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.852+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:52:31.913 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.909+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.931 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.927+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:31.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.939+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:31.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:31.973+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:52:32.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.018+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.038 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.034+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:32.047 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.043+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.075+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:52:32.129 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.125+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.146+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:32.159 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.155+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.199 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.195+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:52:32.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.237+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.259 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.255+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:32.269 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.265+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.317 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.313+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:52:32.361 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.356+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.373+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:32.389 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.385+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.428+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:52:32.492 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.488+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.511+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:32.525 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.520+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.559 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.555+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:52:32.607 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.602+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.626 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.622+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:32.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.636+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.678 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.674+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:52:32.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.739+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.765 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.761+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:32.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.769+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.818+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:52:32.881 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.877+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.903 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.899+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:32.915 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.911+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:32.953 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:32.949+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:52:33.016 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.007+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.030 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.026+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:33.039 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.035+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.075+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:52:33.138 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.134+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.162 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.158+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:33.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.170+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.221 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.217+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:52:33.269 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.265+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.286 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.282+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:33.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.294+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.341+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:52:33.408 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.404+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.425+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:33.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.437+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.481+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:52:33.569 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.565+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.595 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.591+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:33.610 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.606+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.657 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.653+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:52:33.714 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.710+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.745 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.740+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:33.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.756+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.817 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.812+0000 7f5302796ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:52:33.889 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.884+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:33.916 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.911+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:52:33.934 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:33.930+0000 7f5302796ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:52:34.035 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.031+0000 7f5302796ec0 20 remove_watcher() i=0 2019-11-25T09:52:34.036 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.031+0000 7f5302796ec0 2 removed watcher, disabling cache 2019-11-25T09:52:34.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.114+0000 7f5302796ec0 20 remove_watcher() i=1 2019-11-25T09:52:34.172 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.168+0000 7f5302796ec0 20 remove_watcher() i=2 2019-11-25T09:52:34.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.231+0000 7f5302796ec0 20 remove_watcher() i=3 2019-11-25T09:52:34.288 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.284+0000 7f5302796ec0 20 remove_watcher() i=4 2019-11-25T09:52:34.344 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.340+0000 7f5302796ec0 20 remove_watcher() i=5 2019-11-25T09:52:34.401 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.397+0000 7f5302796ec0 20 remove_watcher() i=6 2019-11-25T09:52:34.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:52:34.453+0000 7f5302796ec0 20 remove_watcher() i=7 2019-11-25T09:52:34.467 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests BucketTest 2019-11-25T09:52:35.291 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :clean 2019-11-25T09:52:36.002 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :compileJava 2019-11-25T09:52:36.003 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :processResources 2019-11-25T09:52:36.003 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :classes 2019-11-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :compileTestJava 2019-11-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi153.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-11-25T09:52:36.290 INFO:teuthology.orchestra.run.smithi153.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-11-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :processTestResources NO-SOURCE 2019-11-25T09:52:36.291 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :testClasses 2019-11-25T09:52:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:52:38.289 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :test 2019-11-25T09:52:38.289 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:52:38.289 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty STARTED 2019-11-25T09:52:40.589 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:52:40.589 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectEmpty PASSED 2019-11-25T09:52:48.089 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:52:48.089 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch STARTED 2019-11-25T09:52:51.289 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:52:51.289 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectMismatch PASSED 2019-11-25T09:52:58.488 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:52:58.489 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable STARTED 2019-11-25T09:53:00.689 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:00.689 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketCreateBadExpectUnreadable PASSED 2019-11-25T09:53:07.488 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:07.489 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete STARTED 2019-11-25T09:53:13.689 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:13.689 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketCreateReadDelete PASSED 2019-11-25T09:53:14.388 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:14.389 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty STARTED 2019-11-25T09:53:19.089 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:19.089 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNonEmpty PASSED 2019-11-25T09:53:26.888 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:26.889 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist STARTED 2019-11-25T09:53:26.989 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:26.989 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketDeleteNotExist PASSED 2019-11-25T09:53:28.688 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:28.689 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct STARTED 2019-11-25T09:53:34.488 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:34.489 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketListDistinct PASSED 2019-11-25T09:53:46.488 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:46.489 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty STARTED 2019-11-25T09:53:50.589 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:50.589 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketListEmpty PASSED 2019-11-25T09:53:58.288 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:58.289 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketNotExist STARTED 2019-11-25T09:53:58.488 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:58.489 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Bucket tests > BucketTest.testBucketNotExist PASSED 2019-11-25T09:53:59.331 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:59.331 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------- 2019-11-25T09:53:59.331 INFO:teuthology.orchestra.run.smithi153.stdout:| Results: SUCCESS (9 tests, 9 successes, 0 failures, 0 skipped) | 2019-11-25T09:53:59.331 INFO:teuthology.orchestra.run.smithi153.stdout:-------------------------------------------------------------------- 2019-11-25T09:53:59.332 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:53:59.332 INFO:teuthology.orchestra.run.smithi153.stdout:BUILD SUCCESSFUL in 1m 24s 2019-11-25T09:53:59.332 INFO:teuthology.orchestra.run.smithi153.stdout:5 actionable tasks: 5 executed 2019-11-25T09:53:59.338 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:53:59.463 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.460+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.499 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.496+0000 7fe25c28cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:53:59.499 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.496+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.509 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.506+0000 7fe25c28cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:53:59.509 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.506+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.514+0000 7fe25c28cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:53:59.517 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.514+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.536 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.533+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:53:59.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.806+0000 7fe25c28cec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.806+0000 7fe25c28cec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.806+0000 7fe25c28cec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.806+0000 7fe25c28cec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:53:59.810 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.806+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.820+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:53:59.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.820+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.831+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:53:59.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.831+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.847 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.843+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:53:59.847 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.843+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.855+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:53:59.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.855+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.864 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.861+0000 7fe25c28cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:53:59.864 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.861+0000 7fe25c28cec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:53:59.864 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.861+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.865+0000 7fe25c28cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:53:59.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.865+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.879 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.876+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:53:59.880 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.876+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.889 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.886+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:53:59.889 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.886+0000 7fe25c28cec0 20 zonegroup default 2019-11-25T09:53:59.889 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.886+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.894+0000 7fe25c28cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:53:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.894+0000 7fe25c28cec0 10 Cannot find current period zone using local zone 2019-11-25T09:53:59.898 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.894+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.900+0000 7fe25c28cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:53:59.903 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.900+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.911 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.907+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:53:59.911 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.908+0000 7fe25c28cec0 20 rados->read ofs=0 len=0 2019-11-25T09:53:59.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.919+0000 7fe25c28cec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:53:59.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.919+0000 7fe25c28cec0 20 zone default found 2019-11-25T09:53:59.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.919+0000 7fe25c28cec0 20 started sync module instance, tier type = 2019-11-25T09:53:59.923 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:53:59.919+0000 7fe25c28cec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:54:00.298 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.295+0000 7fe25c28cec0 20 add_watcher() i=0 2019-11-25T09:54:00.298 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.295+0000 7fe25c28cec0 20 add_watcher() i=1 2019-11-25T09:54:00.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.295+0000 7fe25c28cec0 20 add_watcher() i=2 2019-11-25T09:54:00.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.295+0000 7fe25c28cec0 20 add_watcher() i=3 2019-11-25T09:54:00.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.295+0000 7fe25c28cec0 20 add_watcher() i=4 2019-11-25T09:54:00.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.295+0000 7fe25c28cec0 20 add_watcher() i=5 2019-11-25T09:54:00.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.295+0000 7fe25c28cec0 20 add_watcher() i=6 2019-11-25T09:54:00.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.330+0000 7fe25c28cec0 20 add_watcher() i=7 2019-11-25T09:54:00.334 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.330+0000 7fe25c28cec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:54:00.334 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.330+0000 7fe219fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:54:00.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.337+0000 7fe249cbb700 0 -- 172.21.15.153:0/1030908705 >> v2:172.21.15.81:3300/0 conn(0x561b36593580 msgr2=0x561b365928a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-11-25T09:54:00.542 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.539+0000 7fe249cbb700 0 --2- 172.21.15.153:0/1030908705 >> v2:172.21.15.81:3300/0 conn(0x561b36593580 0x561b365928a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-25T09:54:00.588 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:00.585+0000 7fe249cbb700 0 -- 172.21.15.153:0/3616316975 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7fe18c017570 msgr2=0x7fe18c019a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T09:54:02.072 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.069+0000 7fe18bfff700 20 reqs_thread_entry: start 2019-11-25T09:54:02.073 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.070+0000 7fe189ffb700 20 reqs_thread_entry: start 2019-11-25T09:54:02.073 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.070+0000 7fe25c28cec0 20 init_complete bucket index max shards: 0 2019-11-25T09:54:02.074 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.071+0000 7fe177fff700 20 reqs_thread_entry: start 2019-11-25T09:54:02.074 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.071+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:54:02.155 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.151+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.183+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:02.200 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.197+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.251 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.247+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:54:02.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.342+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.395 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.392+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:02.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.410+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.462+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:54:02.524 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.521+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.542+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:02.555 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.552+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.595 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.592+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:54:02.652 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.649+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.673 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.670+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:02.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.681+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.722 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.715+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:54:02.765 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.761+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.788+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:02.805 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.802+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.867 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.864+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:54:02.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.927+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:02.953 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.949+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:02.964 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:02.961+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.009 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.006+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:54:03.075 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.072+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.097 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.094+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:03.108 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.105+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.160 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.156+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:54:03.252 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.249+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.283 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.280+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:03.295 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.292+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.342+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:54:03.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.428+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.469 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.466+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:03.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.486+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.536 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.533+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:54:03.597 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.594+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.617 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.614+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:03.627 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.624+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.667 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.663+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:54:03.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.726+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.746+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:03.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.756+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.800 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.797+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:54:03.867 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.864+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.889 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.886+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:03.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.903+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:03.961 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:03.958+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:54:04.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.010+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.034 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.030+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:04.045 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.042+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.076+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:54:04.131 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.128+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.148 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.145+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:04.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.154+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.189 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.186+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:54:04.269 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.265+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.295 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.291+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:04.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.300+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.358+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:54:04.446 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.443+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.475 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.470+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:04.487 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.484+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.520+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:54:04.578 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.575+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.595 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.592+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:04.604 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.601+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.637 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.634+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:54:04.678 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.675+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.694 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.691+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:04.702 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.699+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.740 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.737+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:54:04.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.775+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.791+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:04.803 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.800+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.832 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.829+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:54:04.890 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.887+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.913+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:04.932 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.929+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:04.998 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:04.995+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:54:05.058 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.055+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.078+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:05.093 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.090+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.142 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.139+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:54:05.206 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.203+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.221+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:05.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.230+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.282 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.279+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:54:05.361 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.358+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.388 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.385+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:05.399 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.396+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.446 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.443+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:54:05.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.505+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.528 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.525+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:05.536 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.533+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.576+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:54:05.632 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.628+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.651 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.648+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:05.664 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.661+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.711 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.704+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:54:05.750 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.747+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.768 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.765+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:05.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.772+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.819 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.816+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:54:05.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.857+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.880 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.877+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:05.888 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.885+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.922 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.919+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:54:05.963 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.960+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:05.981 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.978+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:05.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:05.987+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.021 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.018+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:54:06.069 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.066+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.091+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:06.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.100+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.144 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.141+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:54:06.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.192+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.212 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.209+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:06.221 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.218+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.272+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:54:06.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.329+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.357 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.354+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:06.368 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.365+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.420 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.417+0000 7fe25c28cec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:54:06.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.492+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.527 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.524+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:54:06.539 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.536+0000 7fe25c28cec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:54:06.622 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.619+0000 7fe25c28cec0 20 remove_watcher() i=0 2019-11-25T09:54:06.623 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.619+0000 7fe25c28cec0 2 removed watcher, disabling cache 2019-11-25T09:54:06.659 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.656+0000 7fe25c28cec0 20 remove_watcher() i=1 2019-11-25T09:54:06.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.695+0000 7fe25c28cec0 20 remove_watcher() i=2 2019-11-25T09:54:06.745 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.741+0000 7fe25c28cec0 20 remove_watcher() i=3 2019-11-25T09:54:06.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.782+0000 7fe25c28cec0 20 remove_watcher() i=4 2019-11-25T09:54:06.827 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.824+0000 7fe25c28cec0 20 remove_watcher() i=5 2019-11-25T09:54:06.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.854+0000 7fe25c28cec0 20 remove_watcher() i=6 2019-11-25T09:54:06.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:06.888+0000 7fe25c28cec0 20 remove_watcher() i=7 2019-11-25T09:54:06.898 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:54:07.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.001+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.030 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.027+0000 7f6f6abe5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:54:07.030 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.027+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.040 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.037+0000 7f6f6abe5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:54:07.041 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.037+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.049 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.046+0000 7f6f6abe5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:54:07.049 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.046+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.067+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:54:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.272+0000 7f6f6abe5ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:54:07.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.272+0000 7f6f6abe5ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:54:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.272+0000 7f6f6abe5ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:54:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.272+0000 7f6f6abe5ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:54:07.276 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.272+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.290+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:54:07.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.290+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.300+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:54:07.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.300+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.318 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.315+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:54:07.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.315+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.334 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.331+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:54:07.334 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.331+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.342+0000 7f6f6abe5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:54:07.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.342+0000 7f6f6abe5ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:54:07.346 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.342+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.356 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.353+0000 7f6f6abe5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:54:07.357 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.353+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.368+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:54:07.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.368+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.384 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.381+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:54:07.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.381+0000 7f6f6abe5ec0 20 zonegroup default 2019-11-25T09:54:07.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.381+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.392 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.389+0000 7f6f6abe5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:54:07.392 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.389+0000 7f6f6abe5ec0 10 Cannot find current period zone using local zone 2019-11-25T09:54:07.393 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.389+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.399+0000 7f6f6abe5ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:54:07.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.399+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.415 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.412+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:54:07.415 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.412+0000 7f6f6abe5ec0 20 rados->read ofs=0 len=0 2019-11-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.432+0000 7f6f6abe5ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.432+0000 7f6f6abe5ec0 20 zone default found 2019-11-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.432+0000 7f6f6abe5ec0 20 started sync module instance, tier type = 2019-11-25T09:54:07.435 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.432+0000 7f6f6abe5ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:54:07.792 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.788+0000 7f6f6abe5ec0 20 add_watcher() i=0 2019-11-25T09:54:07.792 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.788+0000 7f6f6abe5ec0 20 add_watcher() i=1 2019-11-25T09:54:07.792 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.788+0000 7f6f6abe5ec0 20 add_watcher() i=2 2019-11-25T09:54:07.792 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.788+0000 7f6f6abe5ec0 20 add_watcher() i=3 2019-11-25T09:54:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.788+0000 7f6f6abe5ec0 20 add_watcher() i=4 2019-11-25T09:54:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.788+0000 7f6f6abe5ec0 20 add_watcher() i=5 2019-11-25T09:54:07.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.788+0000 7f6f6abe5ec0 20 add_watcher() i=6 2019-11-25T09:54:07.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.820+0000 7f6f6abe5ec0 20 add_watcher() i=7 2019-11-25T09:54:07.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.820+0000 7f6f6abe5ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:54:07.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.820+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:54:07.831 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:07.828+0000 7f6f58614700 0 -- 172.21.15.153:0/3906000495 >> v1:172.21.15.153:6789/0 conn(0x56118d213b50 legacy=0x56118d22c280 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:54:17.036 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:17.032+0000 7f6f57e13700 0 -- 172.21.15.153:0/376060867 >> [v2:172.21.15.153:6804/9693,v1:172.21.15.153:6805/9693] conn(0x7f6f4001b090 msgr2=0x7f6f4001b4a0 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:54:29.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:29.820+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:54:32.080 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T09:54:32.072+0000 28256700 -1 reset not still connected to 0x3badc9f0 2019-11-25T09:54:51.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:54:51.820+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:55:13.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:55:13.820+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:55:35.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:55:35.820+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:55:57.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:55:57.819+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:56:19.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:56:19.820+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:56:41.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:56:41.820+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:57:03.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:57:03.820+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:57:16.799 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:42:21.379 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:16.810 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:42:21.391 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:16.814 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:42:21.419 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:16.818 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:42:21.424 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:16.820 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:42:21.425 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:16.827 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:42:21.432 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:17.023 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:42:21.604 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:17.029 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:42:21.634 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:17.037 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:42:21.642 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:25.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:57:25.821+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:57:29.570 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:42:34.151 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:29.580 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:42:34.160 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:29.609 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:42:34.190 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:29.618 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:42:34.223 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:29.618 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:42:34.224 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:29.634 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:42:34.239 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:29.634 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:42:34.240 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:29.658 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:42:34.264 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:29.660 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:42:34.265 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T09:57:47.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:57:47.821+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:58:09.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:58:09.821+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:58:31.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:58:31.821+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:58:53.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:58:53.821+0000 7f6eabfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:59:09.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.694+0000 7f6e8e7fc700 20 reqs_thread_entry: start 2019-11-25T09:59:09.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.694+0000 7f6e83fff700 20 reqs_thread_entry: start 2019-11-25T09:59:09.698 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.694+0000 7f6f6abe5ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:59:09.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.695+0000 7f6e827fc700 20 reqs_thread_entry: start 2019-11-25T09:59:09.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.695+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:59:09.765 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.761+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:09.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.790+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:09.810 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.807+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:09.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.853+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:59:09.925 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.921+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:09.943 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.939+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:09.955 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.951+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:09.997 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:09.993+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:59:10.048 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.043+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.065 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.060+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:10.073 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.069+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.111 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.106+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:59:10.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.190+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.225 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.220+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:10.237 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.232+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.295 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.281+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:59:10.349 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.345+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.375 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.370+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:10.386 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.382+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.424+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:59:10.506 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.501+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.533 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.528+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:10.542 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.537+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.598 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.593+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:59:10.673 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.669+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.700 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.695+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:10.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.705+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.776 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.772+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:59:10.826 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.821+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.844 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.840+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:10.854 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.849+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.895 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.890+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:59:10.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.935+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:10.961 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.956+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:10.971 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:10.966+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.007 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.003+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:59:11.044 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.040+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.059 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.055+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:11.067 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.063+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.103 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.098+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:59:11.212 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.207+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.231+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:11.249 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.244+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.286+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:59:11.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.365+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.401 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.396+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:11.417 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.412+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.463 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.458+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:59:11.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.541+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.582 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.578+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:11.599 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.594+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.664 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.659+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:59:11.746 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.741+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.779 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.775+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:11.796 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.792+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.861 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.856+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:59:11.934 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.930+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:11.957 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.952+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:11.980 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:11.975+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.030 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.025+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:59:12.109 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.104+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.126 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.121+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:12.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.130+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.171 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.166+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:59:12.215 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.211+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.228+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:12.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.238+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.275 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.270+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:59:12.358 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.353+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.386 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.381+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:12.400 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.396+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.443 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.438+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:59:12.498 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.494+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.533+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:12.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.542+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.587 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.582+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:59:12.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.679+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.714 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.710+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:12.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.724+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.774 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.769+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:59:12.840 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.835+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.855+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:12.870 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.866+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.904 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.899+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:59:12.950 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.946+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:12.966 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.962+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:12.976 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:12.972+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.006 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.002+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:59:13.052 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.048+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.067 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.063+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:13.076 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.072+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.113+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:59:13.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.165+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.189 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.185+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:13.200 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.195+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.234+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:59:13.316 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.311+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.358 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.354+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:13.376 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.371+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.443 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.438+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:59:13.517 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.513+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.542+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:13.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.558+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.611+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:59:13.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.677+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.697+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:13.715 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.710+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.753 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.748+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:59:13.809 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.804+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.837 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.832+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:13.849 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.845+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.901 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.896+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:59:13.953 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.948+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:13.977 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.972+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:13.987 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:13.982+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:14.020 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.015+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:59:14.080 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.075+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:14.101 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.097+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:14.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.110+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:14.154 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.150+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:59:14.228 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.223+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:14.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.248+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:14.266 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.261+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.304+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:59:14.378 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.373+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:14.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.399+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:14.416 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.412+0000 7f6f6abe5ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:14.529 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.525+0000 7f6f6abe5ec0 20 remove_watcher() i=0 2019-11-25T09:59:14.529 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.525+0000 7f6f6abe5ec0 2 removed watcher, disabling cache 2019-11-25T09:59:14.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.571+0000 7f6f6abe5ec0 20 remove_watcher() i=1 2019-11-25T09:59:14.628 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.624+0000 7f6f6abe5ec0 20 remove_watcher() i=2 2019-11-25T09:59:14.681 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.677+0000 7f6f6abe5ec0 20 remove_watcher() i=3 2019-11-25T09:59:14.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.735+0000 7f6f6abe5ec0 20 remove_watcher() i=4 2019-11-25T09:59:14.805 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.800+0000 7f6f6abe5ec0 20 remove_watcher() i=5 2019-11-25T09:59:14.850 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.846+0000 7f6f6abe5ec0 20 remove_watcher() i=6 2019-11-25T09:59:14.891 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:14.886+0000 7f6f6abe5ec0 20 remove_watcher() i=7 2019-11-25T09:59:14.899 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:59:15.039 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.035+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.066 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.062+0000 7fbba418eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:15.066 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.062+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.070+0000 7fbba418eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:15.079 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.070+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.085 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.080+0000 7fbba418eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:15.085 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.080+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.107 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.103+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:15.316 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.311+0000 7fbba418eec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:59:15.316 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.311+0000 7fbba418eec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:59:15.316 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.311+0000 7fbba418eec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:59:15.317 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.311+0000 7fbba418eec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:59:15.317 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.311+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.331 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.326+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:15.331 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.326+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.334+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:59:15.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.335+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.359 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.355+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:15.359 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.355+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.376 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.371+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:59:15.376 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.371+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.384 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.379+0000 7fbba418eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:15.384 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.379+0000 7fbba418eec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:59:15.384 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.379+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.393 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.388+0000 7fbba418eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:15.393 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.388+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.405+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:15.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.405+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.419+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:59:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.419+0000 7fbba418eec0 20 zonegroup default 2019-11-25T09:59:15.424 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.419+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.428+0000 7fbba418eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:15.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.428+0000 7fbba418eec0 10 Cannot find current period zone using local zone 2019-11-25T09:59:15.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.428+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.436+0000 7fbba418eec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:15.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.436+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.448+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:15.453 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.448+0000 7fbba418eec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:15.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.460+0000 7fbba418eec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:59:15.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.460+0000 7fbba418eec0 20 zone default found 2019-11-25T09:59:15.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.460+0000 7fbba418eec0 20 started sync module instance, tier type = 2019-11-25T09:59:15.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.460+0000 7fbba418eec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:59:15.904 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.900+0000 7fbba418eec0 20 add_watcher() i=0 2019-11-25T09:59:15.905 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.900+0000 7fbba418eec0 20 add_watcher() i=1 2019-11-25T09:59:15.905 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.900+0000 7fbba418eec0 20 add_watcher() i=2 2019-11-25T09:59:15.905 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.900+0000 7fbba418eec0 20 add_watcher() i=3 2019-11-25T09:59:15.905 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.900+0000 7fbba418eec0 20 add_watcher() i=4 2019-11-25T09:59:15.905 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.900+0000 7fbba418eec0 20 add_watcher() i=5 2019-11-25T09:59:15.905 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.900+0000 7fbba418eec0 20 add_watcher() i=6 2019-11-25T09:59:15.987 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.983+0000 7fbba418eec0 20 add_watcher() i=7 2019-11-25T09:59:15.987 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.983+0000 7fbba418eec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:59:15.988 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:15.983+0000 7fbb61fdb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:59:16.014 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:16.009+0000 7fbb88bbb700 0 -- 172.21.15.153:0/1405915762 >> v2:172.21.15.81:3300/0 conn(0x5558b2da6e30 msgr2=0x5558b2de6230 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:59:16.057 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:16.053+0000 7fbb913bc700 0 -- 172.21.15.153:0/1699633930 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7fbad40175a0 msgr2=0x7fbad4019a90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:59:17.506 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.500+0000 7fbad3fff700 20 reqs_thread_entry: start 2019-11-25T09:59:17.506 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.501+0000 7fbad1ffb700 20 reqs_thread_entry: start 2019-11-25T09:59:17.506 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.501+0000 7fbba418eec0 20 init_complete bucket index max shards: 0 2019-11-25T09:59:17.507 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.502+0000 7fbabffff700 20 reqs_thread_entry: start 2019-11-25T09:59:17.507 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.502+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:59:17.589 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.584+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:17.618 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.613+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:17.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.630+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:17.690 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.685+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:59:17.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.786+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:17.823 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.819+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:17.835 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.830+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:17.880 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.875+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:59:17.959 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.956+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:17.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.975+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:17.987 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:17.983+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.033 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.030+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:59:18.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.109+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.142 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.138+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:18.154 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.150+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.191 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.187+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:59:18.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.239+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.266 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.262+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:18.285 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.281+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.336 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.332+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:59:18.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.381+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.407 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.404+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:18.416 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.413+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.448 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.445+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:59:18.501 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.498+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.520+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:18.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.529+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.559+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:59:18.618 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.614+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.646 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.631+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:18.646 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.642+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.677 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.674+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:59:18.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.724+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.744 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.741+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:18.760 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.757+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.788+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:59:18.837 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.834+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.853+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:18.867 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.863+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.899 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.895+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:59:18.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.940+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.959 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.956+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:18.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.964+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:18.998 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:18.995+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:59:19.042 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.039+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.062 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.059+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:19.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.067+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.106 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.103+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:59:19.144 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.141+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.160 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.157+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:19.168 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.165+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.197 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.194+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:59:19.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.233+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.251+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:19.263 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.260+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.294 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.291+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:59:19.332 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.329+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.347 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.344+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:19.356 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.352+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.385 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.381+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:59:19.456 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.452+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.486 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.483+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:19.501 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.497+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.548+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:59:19.611 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.608+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.636 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.633+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:19.656 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.652+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.714 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.711+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:59:19.761 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.758+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.777 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.774+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:19.787 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.783+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.832 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.828+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:59:19.904 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.900+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.934 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.931+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:19.950 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.946+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:19.995 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:19.992+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:59:20.067 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.063+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.101 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.098+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:20.110 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.107+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.145 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.141+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:59:20.185 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.182+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.202 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.199+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:20.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.208+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.260 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.256+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:59:20.314 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.310+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.331 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.327+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:20.337 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.334+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.378 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.375+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:59:20.437 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.433+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.462+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:20.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.480+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.518+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:59:20.570 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.566+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.592 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.588+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:20.601 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.598+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.645 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.642+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:59:20.687 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.683+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.704 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.701+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:20.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.710+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.745 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.741+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:59:20.789 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.785+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.805 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.801+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:20.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.809+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.855 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.851+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:59:20.916 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.912+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.937 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.934+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:20.948 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.944+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:20.976 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:20.972+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:59:21.021 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.018+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.033+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:21.044 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.040+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.084 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.081+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:59:21.138 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.133+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.159 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.155+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:21.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.167+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.213 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.209+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:59:21.282 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.278+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.308 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.304+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:21.320 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.317+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.375 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.371+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:59:21.443 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.439+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.462 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.458+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:21.471 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.468+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.517 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.510+0000 7fbba418eec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:59:21.580 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.577+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.605+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:21.623 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.619+0000 7fbba418eec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:21.719 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.715+0000 7fbba418eec0 20 remove_watcher() i=0 2019-11-25T09:59:21.719 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.715+0000 7fbba418eec0 2 removed watcher, disabling cache 2019-11-25T09:59:21.788 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.785+0000 7fbba418eec0 20 remove_watcher() i=1 2019-11-25T09:59:21.855 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.852+0000 7fbba418eec0 20 remove_watcher() i=2 2019-11-25T09:59:21.912 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.909+0000 7fbba418eec0 20 remove_watcher() i=3 2019-11-25T09:59:21.970 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:21.967+0000 7fbba418eec0 20 remove_watcher() i=4 2019-11-25T09:59:22.065 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.061+0000 7fbba418eec0 20 remove_watcher() i=5 2019-11-25T09:59:22.115 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.111+0000 7fbba418eec0 20 remove_watcher() i=6 2019-11-25T09:59:22.154 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.151+0000 7fbba418eec0 20 remove_watcher() i=7 2019-11-25T09:59:22.163 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T09:59:22.272 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.268+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.306+0000 7f70518f2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:22.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.306+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.315+0000 7f70518f2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:22.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.315+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.326 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.322+0000 7f70518f2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:22.326 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.322+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.343 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.340+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:22.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.546+0000 7f70518f2ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T09:59:22.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.546+0000 7f70518f2ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T09:59:22.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.546+0000 7f70518f2ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T09:59:22.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.546+0000 7f70518f2ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T09:59:22.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.546+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.561+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:22.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.561+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.582 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.578+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:59:22.582 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.578+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.593 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.589+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:22.594 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.589+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.609+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:59:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.609+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.619+0000 7f70518f2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:22.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.619+0000 7f70518f2ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T09:59:22.625 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.619+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.633 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.630+0000 7f70518f2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:22.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.630+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.646+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:22.655 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.646+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.676 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.662+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T09:59:22.676 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.662+0000 7f70518f2ec0 20 zonegroup default 2019-11-25T09:59:22.677 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.662+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.674+0000 7f70518f2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:22.685 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.674+0000 7f70518f2ec0 10 Cannot find current period zone using local zone 2019-11-25T09:59:22.685 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.674+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.692 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.685+0000 7f70518f2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T09:59:22.692 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.685+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.697+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T09:59:22.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.697+0000 7f70518f2ec0 20 rados->read ofs=0 len=0 2019-11-25T09:59:22.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.707+0000 7f70518f2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T09:59:22.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.707+0000 7f70518f2ec0 20 zone default found 2019-11-25T09:59:22.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.707+0000 7f70518f2ec0 20 started sync module instance, tier type = 2019-11-25T09:59:22.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:22.707+0000 7f70518f2ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T09:59:23.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.000+0000 7f70518f2ec0 20 add_watcher() i=0 2019-11-25T09:59:23.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.000+0000 7f70518f2ec0 20 add_watcher() i=1 2019-11-25T09:59:23.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.000+0000 7f70518f2ec0 20 add_watcher() i=2 2019-11-25T09:59:23.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.000+0000 7f70518f2ec0 20 add_watcher() i=3 2019-11-25T09:59:23.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.000+0000 7f70518f2ec0 20 add_watcher() i=4 2019-11-25T09:59:23.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.000+0000 7f70518f2ec0 20 add_watcher() i=5 2019-11-25T09:59:23.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.000+0000 7f70518f2ec0 20 add_watcher() i=6 2019-11-25T09:59:23.034 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.030+0000 7f70518f2ec0 20 add_watcher() i=7 2019-11-25T09:59:23.034 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.030+0000 7f70518f2ec0 2 all 8 watchers are set, enabling cache 2019-11-25T09:59:23.034 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.030+0000 7f700f7de700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T09:59:23.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.033+0000 7f703e31f700 0 -- 172.21.15.153:0/177641618 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x562fc3cd8b90 msgr2=0x562fc3cdf510 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T09:59:23.069 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:23.066+0000 7f703eb20700 0 -- 172.21.15.153:0/2459542631 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7f6f800175a0 msgr2=0x7f6f80019a90 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T09:59:24.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.853+0000 7f6f7dffb700 20 reqs_thread_entry: start 2019-11-25T09:59:24.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.854+0000 7f6f6f7fe700 20 reqs_thread_entry: start 2019-11-25T09:59:24.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.854+0000 7f70518f2ec0 20 init_complete bucket index max shards: 0 2019-11-25T09:59:24.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.854+0000 7f6f6dffb700 20 reqs_thread_entry: start 2019-11-25T09:59:24.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.854+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T09:59:24.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.911+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:24.942 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.938+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:24.956 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.953+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:24.989 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:24.985+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T09:59:25.035 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.032+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.054 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.051+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:25.066 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.062+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.098 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.095+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T09:59:25.143 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.139+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.163+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:25.177 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.173+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.211+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T09:59:25.281 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.277+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.306 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.303+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:25.320 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.316+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.361 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.357+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T09:59:25.432 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.429+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.456 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.452+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:25.474 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.470+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.522 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.519+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T09:59:25.685 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.681+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.709 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.706+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:25.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.725+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.779 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.776+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T09:59:25.849 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.846+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.877 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.873+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:25.894 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.890+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:25.947 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:25.943+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T09:59:26.055 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.052+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.080 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.077+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:26.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.091+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.142 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.138+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T09:59:26.198 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.194+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.217 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.214+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:26.227 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.223+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.267+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T09:59:26.335 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.331+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.360 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.357+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:26.373 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.369+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.406 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.403+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T09:59:26.470 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.466+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.507 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.504+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:26.531 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.527+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.564+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T09:59:26.613 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.610+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.632 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.628+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:26.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.638+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.680 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.677+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T09:59:26.749 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.746+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.777 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.773+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:26.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.791+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.840 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.837+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T09:59:26.903 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.899+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.924 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.920+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:26.934 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.931+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:26.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:26.987+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T09:59:27.051 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.047+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.075 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.072+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:27.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.084+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.136 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.133+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T09:59:27.209 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.206+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.223+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:27.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.232+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.281 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.278+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T09:59:27.362 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.358+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.387 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.383+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:27.395 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.391+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.439 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.436+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T09:59:27.502 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.498+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.519+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:27.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.529+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.572 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.569+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T09:59:27.645 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.642+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.672 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.669+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:27.690 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.686+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.736+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T09:59:27.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.812+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.850 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.846+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:27.867 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.863+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:27.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.917+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T09:59:28.001 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:27.997+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.023 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.020+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:28.035 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.031+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.082 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.079+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T09:59:28.180 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.177+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.208 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.205+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:28.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.221+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.274 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.270+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T09:59:28.336 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.332+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.353 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.349+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:28.368 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.364+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.409 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.406+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T09:59:28.494 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.491+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.531 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.527+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:28.548 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.545+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.613+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T09:59:28.687 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.684+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.723 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.719+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:28.736 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.733+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.787 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.784+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T09:59:28.875 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.871+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.905 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.901+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:28.919 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.916+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:28.975 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:28.971+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T09:59:29.040 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.037+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.063 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.060+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:29.075 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.071+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.120 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.117+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T09:59:29.165 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.162+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.190 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.187+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:29.202 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.199+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.243 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.240+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T09:59:29.315 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.312+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.339+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:29.354 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.351+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.408 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.405+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T09:59:29.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.486+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.519 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.515+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:29.534 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.531+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.591 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.588+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T09:59:29.677 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.673+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.699 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.695+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:29.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.706+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.758 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.755+0000 7f70518f2ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T09:59:29.846 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.842+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.861 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.858+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T09:59:29.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.866+0000 7f70518f2ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T09:59:29.947 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.944+0000 7f70518f2ec0 20 remove_watcher() i=0 2019-11-25T09:59:29.947 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.944+0000 7f70518f2ec0 2 removed watcher, disabling cache 2019-11-25T09:59:29.994 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:29.991+0000 7f70518f2ec0 20 remove_watcher() i=1 2019-11-25T09:59:30.046 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:30.042+0000 7f70518f2ec0 20 remove_watcher() i=2 2019-11-25T09:59:30.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:30.083+0000 7f70518f2ec0 20 remove_watcher() i=3 2019-11-25T09:59:30.131 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:30.127+0000 7f70518f2ec0 20 remove_watcher() i=4 2019-11-25T09:59:30.181 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:30.177+0000 7f70518f2ec0 20 remove_watcher() i=5 2019-11-25T09:59:30.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:30.233+0000 7f70518f2ec0 20 remove_watcher() i=6 2019-11-25T09:59:30.278 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T09:59:30.275+0000 7f70518f2ec0 20 remove_watcher() i=7 2019-11-25T09:59:30.287 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest/s3-tests-java && /opt/gradle/gradle-4.7/bin/gradle clean test --rerun-tasks --no-build-cache --tests ObjectTest 2019-11-25T09:59:30.810 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :clean 2019-11-25T09:59:31.306 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :compileJava 2019-11-25T09:59:31.306 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :processResources 2019-11-25T09:59:31.306 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :classes 2019-11-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:59:31.507 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :compileTestJava 2019-11-25T09:59:31.508 INFO:teuthology.orchestra.run.smithi153.stdout:Note: /home/ubuntu/cephtest/s3-tests-java/src/test/java/BucketTest.java uses or overrides a deprecated API. 2019-11-25T09:59:31.508 INFO:teuthology.orchestra.run.smithi153.stdout:Note: Recompile with -Xlint:deprecation for details. 2019-11-25T09:59:31.508 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:59:31.508 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :processTestResources NO-SOURCE 2019-11-25T09:59:31.508 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :testClasses 2019-11-25T09:59:34.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:59:34.006 INFO:teuthology.orchestra.run.smithi153.stdout:> Task :test 2019-11-25T09:59:34.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:59:34.007 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound STARTED 2019-11-25T09:59:37.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:59:37.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.TestObjectCopyKeyNotFound PASSED 2019-11-25T09:59:50.608 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:59:50.608 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone STARTED 2019-11-25T09:59:54.106 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T09:59:54.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadExpectNone PASSED 2019-11-25T10:00:03.306 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:00:03.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty STARTED 2019-11-25T10:00:04.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:00:04.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.TestObjectCreateBadMd5Empty PASSED 2019-11-25T10:00:14.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:00:14.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI STARTED 2019-11-25T10:00:58.206 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:00:58.206 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadLLAPI PASSED 2019-11-25T10:01:05.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:05.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI STARTED 2019-11-25T10:01:08.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:08.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testAbortMultipartUploadNotFoundLLAPI PASSED 2019-11-25T10:01:14.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:14.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI STARTED 2019-11-25T10:01:20.406 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:20.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testDownloadHLAPI PASSED 2019-11-25T10:01:30.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:30.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI STARTED 2019-11-25T10:01:30.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:30.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testDownloadNoBucketHLAPI PASSED 2019-11-25T10:01:31.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:31.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI STARTED 2019-11-25T10:01:33.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:33.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testDownloadNoKeyHLAPI PASSED 2019-11-25T10:01:41.808 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:41.808 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB STARTED 2019-11-25T10:01:49.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:49.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1MB PASSED 2019-11-25T10:01:57.106 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:01:57.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b STARTED 2019-11-25T10:02:00.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:00.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1b PASSED 2019-11-25T10:02:10.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:10.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb STARTED 2019-11-25T10:02:13.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:13.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptedTransfer1kb PASSED 2019-11-25T10:02:23.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:23.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC STARTED 2019-11-25T10:02:25.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:25.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptionKeyNoSSEC PASSED 2019-11-25T10:02:27.278 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:47:31.858 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:27.287 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:47:31.868 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:27.291 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:47:31.897 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:27.296 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:47:31.902 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:27.298 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:47:31.904 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:27.305 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:47:31.910 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:27.464 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:47:32.045 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:27.465 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:47:32.071 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:27.505 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:47:32.111 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:02:31.406 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:31.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 STARTED 2019-11-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:34.206 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECInvalidMd5 PASSED 2019-11-25T10:02:42.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:42.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey STARTED 2019-11-25T10:02:46.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:46.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoKey PASSED 2019-11-25T10:02:53.405 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:53.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 STARTED 2019-11-25T10:02:55.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:02:55.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testEncryptionKeySSECNoMd5 PASSED 2019-11-25T10:03:01.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:03:01.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete STARTED 2019-11-25T10:03:06.005 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:03:06.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultiObjectDelete PASSED 2019-11-25T10:03:13.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:03:13.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA STARTED 2019-11-25T10:03:32.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:03:32.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartCopyHLAPIA PASSED 2019-11-25T10:03:48.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:03:48.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI STARTED 2019-11-25T10:05:51.367 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:50:55.948 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:05:51.384 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:50:55.965 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:05:51.421 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:50:56.026 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:05:51.422 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:50:56.028 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:05:51.447 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:50:56.053 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:05:51.455 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:50:56.061 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:05:51.504 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:50:56.085 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:05:51.519 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:50:56.125 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:05:51.541 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:50:56.146 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:30.306 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:07:30.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartCopyMultipleSizesLLAPI PASSED 2019-11-25T10:07:37.782 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:52:42.363 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:37.787 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:52:42.368 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:37.804 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:52:42.410 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:37.807 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:52:42.413 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:37.810 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:52:42.415 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:37.812 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:52:42.418 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:37.902 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:52:42.507 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:37.919 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:52:42.524 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:37.923 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:52:42.504 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:07:43.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:07:43.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI STARTED 2019-11-25T10:07:45.206 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:07:45.206 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCBucketHLAPI PASSED 2019-11-25T10:07:50.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:07:50.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI STARTED 2019-11-25T10:07:55.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:07:55.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartCopyNoSRCKeyHLAPI PASSED 2019-11-25T10:08:06.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:08:06.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI STARTED 2019-11-25T10:08:21.706 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:08:21.706 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartDownloadHLAPI PASSED 2019-11-25T10:08:28.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:08:28.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI STARTED 2019-11-25T10:08:29.306 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:08:29.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartDownloadNoBucketHLAPI PASSED 2019-11-25T10:08:29.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:08:29.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI STARTED 2019-11-25T10:08:44.581 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T10:08:44.521+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T10:08:44.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:08:44.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartDownloadWithPauseHLAPI PASSED 2019-11-25T10:08:53.406 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:08:53.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 STARTED 2019-11-25T10:09:07.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:09:07.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadHLAPIAWS4 PASSED 2019-11-25T10:09:21.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:09:21.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI STARTED 2019-11-25T10:09:47.406 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:09:47.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadIncorrectMissingPartLLAPI PASSED 2019-11-25T10:09:58.106 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:09:58.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI STARTED 2019-11-25T10:12:26.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:12:26.516 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadMultipleSizesLLAPI PASSED 2019-11-25T10:12:37.007 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:12:37.087 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI STARTED 2019-11-25T10:12:51.495 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:57:56.075 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:12:51.505 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:57:56.086 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:12:51.515 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:57:56.120 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:12:51.519 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:57:56.124 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:12:51.526 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:57:56.132 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:12:51.539 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:57:56.144 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:12:51.701 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:57:56.306 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:12:51.746 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:57:56.351 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:12:51.756 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:57:56.337 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:13:09.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:13:09.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadOverwriteExistingObjectLLAPI PASSED 2019-11-25T10:13:18.306 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:13:18.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI STARTED 2019-11-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:13:40.206 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadSmallLLAPI PASSED 2019-11-25T10:13:50.306 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:13:50.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause STARTED 2019-11-25T10:13:58.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:13:58.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testMultipartUploadWithPause PASSED 2019-11-25T10:14:13.009 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:59:17.590 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:13.026 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:59:17.632 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:13.027 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:59:17.608 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:13.042 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:59:17.648 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:13.057 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:59:17.662 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:13.061 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:59:17.666 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:13.094 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:00:59:17.674 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:13.108 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:00:59:17.714 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:13.112 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:00:59:17.718 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:14:19.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:19.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound STARTED 2019-11-25T10:14:19.706 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:19.706 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCopyBucketNotFound PASSED 2019-11-25T10:14:20.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:20.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative STARTED 2019-11-25T10:14:22.412 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:22.412 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadContentlengthNegative PASSED 2019-11-25T10:14:27.106 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:27.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch STARTED 2019-11-25T10:14:28.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:28.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadExpectMismatch PASSED 2019-11-25T10:14:32.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:32.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort STARTED 2019-11-25T10:14:33.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:33.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5InvalidShort PASSED 2019-11-25T10:14:38.206 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:38.206 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid STARTED 2019-11-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:39.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Ivalid PASSED 2019-11-25T10:14:44.306 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:44.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None STARTED 2019-11-25T10:14:45.406 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:45.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5None PASSED 2019-11-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:51.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable STARTED 2019-11-25T10:14:54.106 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:14:54.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectCreateBadMd5Unreadable PASSED 2019-11-25T10:15:00.107 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:15:00.107 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes STARTED 2019-11-25T10:15:02.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:15:02.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectHeadZeroBytes PASSED 2019-11-25T10:15:12.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:15:12.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot STARTED 2019-11-25T10:15:21.707 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:15:21.707 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterDot PASSED 2019-11-25T10:15:34.305 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:15:34.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist STARTED 2019-11-25T10:15:41.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:15:41.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterNotExist PASSED 2019-11-25T10:15:55.905 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:15:55.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage STARTED 2019-11-25T10:16:03.506 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:16:03.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListDelimiterPercentage PASSED 2019-11-25T10:16:14.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:16:14.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList STARTED 2019-11-25T10:16:22.305 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:16:22.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMarkerAfterList PASSED 2019-11-25T10:16:34.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:16:34.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList STARTED 2019-11-25T10:16:41.106 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:16:41.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMarkerBeforeList PASSED 2019-11-25T10:16:52.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:16:52.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList STARTED 2019-11-25T10:16:58.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:16:58.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMarkerNotInList PASSED 2019-11-25T10:17:07.805 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:17:07.806 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable STARTED 2019-11-25T10:17:12.705 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:17:12.706 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMarkerUnreadable PASSED 2019-11-25T10:17:24.205 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:17:24.206 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative STARTED 2019-11-25T10:17:31.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:17:31.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNegative PASSED 2019-11-25T10:17:42.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:17:42.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone STARTED 2019-11-25T10:17:48.006 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:17:48.006 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysNone PASSED 2019-11-25T10:17:59.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:17:59.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne STARTED 2019-11-25T10:18:03.611 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:03:08.192 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:03.619 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:03:08.225 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:03.620 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:03:08.201 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:03.628 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:03:08.231 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:03.628 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:03:08.232 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:03.631 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:03:08.235 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:03.763 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:03:08.368 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:03.782 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:03:08.360 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:03.818 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:03:08.423 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:18:05.605 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:18:05.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysOne PASSED 2019-11-25T10:18:15.606 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:18:15.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero STARTED 2019-11-25T10:18:21.605 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:18:21.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListMaxkeysZero PASSED 2019-11-25T10:18:36.605 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:18:36.606 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt STARTED 2019-11-25T10:18:44.305 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:18:44.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixAlt PASSED 2019-11-25T10:18:54.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:18:54.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic STARTED 2019-11-25T10:19:02.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:19:02.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixBasic PASSED 2019-11-25T10:19:13.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:19:13.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt STARTED 2019-11-25T10:19:20.905 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:19:20.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterAlt PASSED 2019-11-25T10:19:32.305 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:19:32.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic STARTED 2019-11-25T10:19:39.405 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:19:39.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterBasic PASSED 2019-11-25T10:19:52.906 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:19:52.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist STARTED 2019-11-25T10:20:02.305 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:20:02.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterDelimiterNotExist PASSED 2019-11-25T10:20:15.205 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:20:15.205 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist STARTED 2019-11-25T10:20:23.305 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:20:23.306 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixDelimiterNotExist PASSED 2019-11-25T10:20:36.905 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:20:36.905 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist STARTED 2019-11-25T10:20:44.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:20:44.105 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixDelimiterPrefixNotExist PASSED 2019-11-25T10:20:54.805 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:20:54.805 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty STARTED 2019-11-25T10:21:01.405 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:21:01.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixEmpty PASSED 2019-11-25T10:21:13.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:21:13.506 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone STARTED 2019-11-25T10:21:19.905 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:21:19.906 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNone PASSED 2019-11-25T10:21:30.905 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:21:30.905 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist STARTED 2019-11-25T10:21:39.005 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:21:39.005 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixNotExist PASSED 2019-11-25T10:21:50.405 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:21:50.406 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable STARTED 2019-11-25T10:21:58.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:21:58.505 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectListPrefixUnreadable PASSED 2019-11-25T10:22:12.305 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:12.305 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket STARTED 2019-11-25T10:22:14.605 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:14.605 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectReadFromNonExistantBucket PASSED 2019-11-25T10:22:20.705 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:20.705 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist STARTED 2019-11-25T10:22:23.705 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:23.705 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectReadNotExist PASSED 2019-11-25T10:22:30.005 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:30.005 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl STARTED 2019-11-25T10:22:34.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:34.106 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectWriteCacheControl PASSED 2019-11-25T10:22:35.170 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:07:39.751 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:35.174 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:07:39.756 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:35.202 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:07:39.808 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:35.212 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:07:39.818 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:35.212 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:07:39.818 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:35.233 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:07:39.839 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:35.336 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:07:39.918 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:35.364 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:07:39.970 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:35.393 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:07:39.999 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:22:42.805 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:42.805 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag STARTED 2019-11-25T10:22:46.205 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:46.205 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectWriteCheckEtag PASSED 2019-11-25T10:22:56.005 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:22:56.005 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete STARTED 2019-11-25T10:23:05.106 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:05.107 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectWriteReadUpdateReadDelete PASSED 2019-11-25T10:23:12.905 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:12.905 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket STARTED 2019-11-25T10:23:13.605 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:13.605 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testObjectWriteToNonExistantBucket PASSED 2019-11-25T10:23:13.798 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:08:18.380 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:13.806 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:08:18.388 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:13.815 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:08:18.422 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:13.819 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:08:18.425 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:13.820 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:08:18.426 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:13.825 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:08:18.432 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:13.869 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:08:18.475 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:13.880 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:08:18.486 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:13.910 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:08:18.492 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:23:14.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:14.505 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode STARTED 2019-11-25T10:23:17.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:17.505 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testRangedSkipLeadingBytesResponseCode PASSED 2019-11-25T10:23:26.405 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:26.405 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode STARTED 2019-11-25T10:23:30.405 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:30.405 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testRangedrequestResponseCode PASSED 2019-11-25T10:23:39.505 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:39.505 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey STARTED 2019-11-25T10:23:41.705 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:41.705 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testSSEKMSNoKey PASSED 2019-11-25T10:23:47.905 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:47.905 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared STARTED 2019-11-25T10:23:50.005 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:50.005 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testSSEKMSNotDeclared PASSED 2019-11-25T10:23:56.805 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:23:56.805 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent STARTED 2019-11-25T10:24:00.805 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:24:00.805 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testSSEKMSPresent PASSED 2019-11-25T10:24:09.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:24:09.105 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile STARTED 2019-11-25T10:24:36.605 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:24:36.605 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testUploadFileHLAPIBigFile PASSED 2019-11-25T10:24:46.805 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:24:46.805 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI STARTED 2019-11-25T10:25:00.105 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:25:00.105 INFO:teuthology.orchestra.run.smithi153.stdout:suite > Object tests > ObjectTest.testUploadFileListHLAPI PASSED 2019-11-25T10:25:10.302 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:25:10.302 INFO:teuthology.orchestra.run.smithi153.stdout:---------------------------------------------------------------------- 2019-11-25T10:25:10.303 INFO:teuthology.orchestra.run.smithi153.stdout:| Results: SUCCESS (72 tests, 72 successes, 0 failures, 0 skipped) | 2019-11-25T10:25:10.303 INFO:teuthology.orchestra.run.smithi153.stdout:---------------------------------------------------------------------- 2019-11-25T10:25:10.303 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:25:10.303 INFO:teuthology.orchestra.run.smithi153.stdout:BUILD SUCCESSFUL in 25m 39s 2019-11-25T10:25:10.303 INFO:teuthology.orchestra.run.smithi153.stdout:5 actionable tasks: 5 executed 2019-11-25T10:25:10.308 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T10:25:10.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.438+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.479+0000 7f41390f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:10.482 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.479+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.492+0000 7f41390f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:10.495 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.492+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.505+0000 7f41390f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:10.508 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.505+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.530+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:10.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.740+0000 7f41390f9ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T10:25:10.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.740+0000 7f41390f9ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T10:25:10.744 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.740+0000 7f41390f9ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T10:25:10.744 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.740+0000 7f41390f9ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T10:25:10.744 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.740+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.757+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:10.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.757+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.770+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:10.773 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.771+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.783+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:10.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.783+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.795+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:10.798 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.795+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.804 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.802+0000 7f41390f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:10.805 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.802+0000 7f41390f9ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T10:25:10.805 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.802+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.811 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.808+0000 7f41390f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:10.811 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.808+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.822+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:10.824 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.822+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.838 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.836+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:10.838 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.836+0000 7f41390f9ec0 20 zonegroup default 2019-11-25T10:25:10.839 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.836+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.848 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.845+0000 7f41390f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:10.848 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.845+0000 7f41390f9ec0 10 Cannot find current period zone using local zone 2019-11-25T10:25:10.848 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.845+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.859 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.857+0000 7f41390f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:10.859 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.857+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.866+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:10.869 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.866+0000 7f41390f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:10.880 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.877+0000 7f41390f9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:10.880 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.877+0000 7f41390f9ec0 20 zone default found 2019-11-25T10:25:10.880 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.877+0000 7f41390f9ec0 20 started sync module instance, tier type = 2019-11-25T10:25:10.881 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:10.877+0000 7f41390f9ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T10:25:11.211 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.209+0000 7f41390f9ec0 20 add_watcher() i=0 2019-11-25T10:25:11.407 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.209+0000 7f41390f9ec0 20 add_watcher() i=1 2019-11-25T10:25:11.407 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.209+0000 7f41390f9ec0 20 add_watcher() i=2 2019-11-25T10:25:11.408 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.209+0000 7f41390f9ec0 20 add_watcher() i=3 2019-11-25T10:25:11.409 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.209+0000 7f41390f9ec0 20 add_watcher() i=4 2019-11-25T10:25:11.409 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.209+0000 7f41390f9ec0 20 add_watcher() i=5 2019-11-25T10:25:11.409 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.209+0000 7f41390f9ec0 20 add_watcher() i=6 2019-11-25T10:25:11.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.236+0000 7f4126327700 0 -- 172.21.15.153:0/2385022102 >> [v2:172.21.15.153:6804/9693,v1:172.21.15.153:6805/9693] conn(0x7f411000e7b0 msgr2=0x7f4110011050 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T10:25:11.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.236+0000 7f41390f9ec0 20 add_watcher() i=7 2019-11-25T10:25:11.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.236+0000 7f41390f9ec0 2 all 8 watchers are set, enabling cache 2019-11-25T10:25:11.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.236+0000 7f40f6fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:25:11.411 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.255+0000 7f4126327700 0 -- 172.21.15.153:0/3429920678 >> v2:172.21.15.153:3300/0 conn(0x558b855239a0 msgr2=0x558b8552b1e0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T10:25:11.459 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:11.457+0000 7f4126327700 0 --2- 172.21.15.153:0/3429920678 >> v2:172.21.15.153:3300/0 conn(0x558b855239a0 0x558b8552b1e0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=1 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-25T10:25:13.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.274+0000 7f40657fa700 20 reqs_thread_entry: start 2019-11-25T10:25:13.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.274+0000 7f405effd700 20 reqs_thread_entry: start 2019-11-25T10:25:13.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.274+0000 7f41390f9ec0 20 init_complete bucket index max shards: 0 2019-11-25T10:25:13.278 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.275+0000 7f405d7fa700 20 reqs_thread_entry: start 2019-11-25T10:25:13.278 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.275+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T10:25:13.325 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.323+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:13.356 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.353+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:13.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:13.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj', time=2019-11-25T12:09:52.014924+0000, chain.objs.size()=9 2019-11-25T10:25:13.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__multipart_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.7 2019-11-25T10:25:13.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__shadow_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.7_1 2019-11-25T10:25:13.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__multipart_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.8 2019-11-25T10:25:13.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__shadow_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.8_1 2019-11-25T10:25:13.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__multipart_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.9 2019-11-25T10:25:13.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__shadow_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.9_1 2019-11-25T10:25:13.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__multipart_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.10 2019-11-25T10:25:13.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__shadow_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.10_1 2019-11-25T10:25:13.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.399+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.74__multipart_key1.2~sOEWYYqqSPp_FQwMzZUJFacf3NCrEtj.11 2019-11-25T10:25:13.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.756+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:13.906 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.903+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T10:25:13.949 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.946+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:13.968 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.966+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:13.977 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:13.974+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:14.010 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.008+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T10:25:14.077 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.074+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:14.106 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.104+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:14.118 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.115+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:14.173 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.170+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T10:25:14.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.252+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:14.284 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.281+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:14.308 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T10:25:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1154', time=2019-11-25T12:08:48.127353+0000, chain.objs.size()=9 2019-11-25T10:25:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__multipart_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.1 2019-11-25T10:25:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__shadow_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.1_1 2019-11-25T10:25:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__multipart_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.2 2019-11-25T10:25:14.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__shadow_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.2_1 2019-11-25T10:25:14.310 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__multipart_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.3 2019-11-25T10:25:14.310 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__shadow_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.3_1 2019-11-25T10:25:14.310 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__multipart_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.4 2019-11-25T10:25:14.310 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__shadow_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.4_1 2019-11-25T10:25:14.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.72__multipart_key1.2~sf5R0twRszgT2eELRzQ_3jCwrmPfC14.5 2019-11-25T10:25:14.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1905', time=2019-11-25T12:24:39.687447+0000, chain.objs.size()=9 2019-11-25T10:25:14.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__multipart_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.1 2019-11-25T10:25:14.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__shadow_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.1_1 2019-11-25T10:25:14.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.306+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__multipart_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.2 2019-11-25T10:25:14.570 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.567+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__shadow_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.2_1 2019-11-25T10:25:14.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.631+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__multipart_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.3 2019-11-25T10:25:14.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.631+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__shadow_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.3_1 2019-11-25T10:25:14.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.631+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__multipart_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.4 2019-11-25T10:25:14.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.631+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__shadow_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.4_1 2019-11-25T10:25:14.634 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:14.631+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.118__multipart_key1.2~OJcdOgJCHjMRZkmZGNKQO1Ffwevx3uq.5 2019-11-25T10:25:15.244 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.241+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:15.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.479+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T10:25:15.519 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.516+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:15.535 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.531+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:15.551 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T10:25:15.551 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1217', time=2019-11-25T12:10:53.036875+0000, chain.objs.size()=9 2019-11-25T10:25:15.551 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.1 2019-11-25T10:25:15.551 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.1_1 2019-11-25T10:25:15.551 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.2 2019-11-25T10:25:15.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.2_1 2019-11-25T10:25:15.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.3 2019-11-25T10:25:15.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.3_1 2019-11-25T10:25:15.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.4 2019-11-25T10:25:15.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.4_1 2019-11-25T10:25:15.552 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~p7Av6F_9ZPWGlbZg33ZYDeyehX9rmjx.5 2019-11-25T10:25:15.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1276', time=2019-11-25T12:13:42.366111+0000, chain.objs.size()=9 2019-11-25T10:25:15.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__multipart_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.1 2019-11-25T10:25:15.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__shadow_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.1_1 2019-11-25T10:25:15.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.547+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__multipart_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.2 2019-11-25T10:25:15.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.790+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__shadow_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.2_1 2019-11-25T10:25:15.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.791+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__multipart_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.3 2019-11-25T10:25:15.864 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.860+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__shadow_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.3_1 2019-11-25T10:25:15.864 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.860+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__multipart_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.4 2019-11-25T10:25:15.864 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.860+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__shadow_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.4_1 2019-11-25T10:25:15.997 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:15.993+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.77__multipart_key1.2~kjTiTaED2vFzbeqUMeQ_hDpxfFfUG2I.5 2019-11-25T10:25:16.399 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.395+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:16.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.560+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T10:25:16.653 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.644+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:16.687 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.675+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:16.702 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.693+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:16.757 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.747+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T10:25:16.812 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.804+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:16.833 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.826+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:16.841 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.836+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:16.887 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.884+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T10:25:16.936 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.933+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:16.959 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.955+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:16.972 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:16.969+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:17.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.010+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T10:25:17.057 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.054+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:17.074 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.071+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:17.083 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.080+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:17.115 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.112+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T10:25:17.165 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.161+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:17.183 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.179+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:17.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.190+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:17.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.190+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1056', time=2019-11-25T12:07:38.288520+0000, chain.objs.size()=5 2019-11-25T10:25:17.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.190+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.66__shadow_.30huoscmFWWwIcPKM3WgL3wHnRbgEwF_1 2019-11-25T10:25:17.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.190+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.66__shadow_.30huoscmFWWwIcPKM3WgL3wHnRbgEwF_2 2019-11-25T10:25:17.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.190+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.66__shadow_.30huoscmFWWwIcPKM3WgL3wHnRbgEwF_3 2019-11-25T10:25:17.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.190+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.66__shadow_.30huoscmFWWwIcPKM3WgL3wHnRbgEwF_4 2019-11-25T10:25:17.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.190+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.66__shadow_.30huoscmFWWwIcPKM3WgL3wHnRbgEwF_5 2019-11-25T10:25:17.455 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.451+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:17.631 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.627+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T10:25:17.710 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.706+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:17.752 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.748+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.779+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T10:25:17.784 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1064', time=2019-11-25T12:05:39.252944+0000, chain.objs.size()=9 2019-11-25T10:25:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.1 2019-11-25T10:25:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.1_1 2019-11-25T10:25:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.2 2019-11-25T10:25:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.2_1 2019-11-25T10:25:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.3 2019-11-25T10:25:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.3_1 2019-11-25T10:25:17.785 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.4 2019-11-25T10:25:17.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.4_1 2019-11-25T10:25:17.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~Mmvh6HstbhV6u_DFvP1di1vqBaPm-6q.5 2019-11-25T10:25:17.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1921', time=2019-11-25T12:25:02.294789+0000, chain.objs.size()=9 2019-11-25T10:25:17.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__multipart_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.1 2019-11-25T10:25:17.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__shadow_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.1_1 2019-11-25T10:25:17.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:17.780+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__multipart_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.2 2019-11-25T10:25:18.057 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.053+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__shadow_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.2_1 2019-11-25T10:25:18.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.109+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__multipart_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.3 2019-11-25T10:25:18.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.109+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__shadow_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.3_1 2019-11-25T10:25:18.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.109+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__multipart_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.4 2019-11-25T10:25:18.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.238+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__shadow_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.4_1 2019-11-25T10:25:18.242 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.238+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.119__multipart_key1/data/file.mpg.2~wKuOtn_044M3TFvTQu7RoGRJuJdNwji.5 2019-11-25T10:25:18.560 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.557+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:18.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.810+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T10:25:18.879 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.876+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:18.905 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.902+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:18.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T10:25:18.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.972', time=2019-11-25T12:01:51.508685+0000, chain.objs.size()=2 2019-11-25T10:25:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.56__shadow_.GEH_6yIBiII0dvrFEoo46vjek2QKTZv_1 2019-11-25T10:25:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.56__shadow_.GEH_6yIBiII0dvrFEoo46vjek2QKTZv_2 2019-11-25T10:25:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1186', time=2019-11-25T12:09:11.633956+0000, chain.objs.size()=9 2019-11-25T10:25:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.1 2019-11-25T10:25:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__shadow_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.1_1 2019-11-25T10:25:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.2 2019-11-25T10:25:18.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__shadow_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.2_1 2019-11-25T10:25:18.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.3 2019-11-25T10:25:18.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__shadow_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.3_1 2019-11-25T10:25:18.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.4 2019-11-25T10:25:18.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__shadow_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.4_1 2019-11-25T10:25:18.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:18.924+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file2.mpg.2~dQLbUbpW0xFwF-ZW9jFn8Lf8r26Dt2N.5 2019-11-25T10:25:19.376 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.372+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:19.724 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.721+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T10:25:19.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.774+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:19.799 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.795+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:19.812 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:19.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0', time=2019-11-25T12:00:56.681262+0000, chain.objs.size()=9 2019-11-25T10:25:19.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__multipart_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.1 2019-11-25T10:25:19.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__shadow_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.1_1 2019-11-25T10:25:19.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__multipart_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.2 2019-11-25T10:25:19.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__shadow_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.2_1 2019-11-25T10:25:19.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__multipart_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.3 2019-11-25T10:25:19.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__shadow_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.3_1 2019-11-25T10:25:19.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__multipart_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.4 2019-11-25T10:25:19.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__shadow_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.4_1 2019-11-25T10:25:19.814 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:19.808+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.52__multipart_key1.2~VYyQbA5jF8roZNbfMHE1Ezt8yyPxFw0.5 2019-11-25T10:25:20.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.273+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:20.369 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.365+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T10:25:20.434 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.431+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:20.471 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.468+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:20.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:20.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1098', time=2019-11-25T12:07:32.322629+0000, chain.objs.size()=7 2019-11-25T10:25:20.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~XiO-Zxgsaax5kcml3kojQN3f56vP811.1 2019-11-25T10:25:20.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~XiO-Zxgsaax5kcml3kojQN3f56vP811.1_1 2019-11-25T10:25:20.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~XiO-Zxgsaax5kcml3kojQN3f56vP811.1_2 2019-11-25T10:25:20.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~XiO-Zxgsaax5kcml3kojQN3f56vP811.2 2019-11-25T10:25:20.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~XiO-Zxgsaax5kcml3kojQN3f56vP811.2_1 2019-11-25T10:25:20.485 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~XiO-Zxgsaax5kcml3kojQN3f56vP811.2_2 2019-11-25T10:25:20.486 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.481+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~XiO-Zxgsaax5kcml3kojQN3f56vP811.3 2019-11-25T10:25:20.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.723+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:20.960 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.956+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T10:25:21.002 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:20.998+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:21.018 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.015+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:21.027 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.023+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:21.059 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.055+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T10:25:21.106 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.103+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:21.123 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.120+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:21.132 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.130+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:21.172 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.169+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T10:25:21.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.212+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:21.230 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.227+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:21.245 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T10:25:21.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1072', time=2019-11-25T12:06:13.703138+0000, chain.objs.size()=9 2019-11-25T10:25:21.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.1 2019-11-25T10:25:21.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.1_1 2019-11-25T10:25:21.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.2 2019-11-25T10:25:21.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.2_1 2019-11-25T10:25:21.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.3 2019-11-25T10:25:21.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.3_1 2019-11-25T10:25:21.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.4 2019-11-25T10:25:21.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.4_1 2019-11-25T10:25:21.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~pJyKVP7_3KnzCr2gmAHE0Vs2lPGB1qB.5 2019-11-25T10:25:21.247 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1231', time=2019-11-25T12:11:53.556611+0000, chain.objs.size()=9 2019-11-25T10:25:21.248 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.1 2019-11-25T10:25:21.248 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.1_1 2019-11-25T10:25:21.248 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.242+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.2 2019-11-25T10:25:21.429 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.426+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.2_1 2019-11-25T10:25:21.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.606+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.3 2019-11-25T10:25:21.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.606+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.3_1 2019-11-25T10:25:21.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.606+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.4 2019-11-25T10:25:21.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.606+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.4_1 2019-11-25T10:25:21.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.606+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~j-kU8Hlpn__93LacL1sOdPyFwCBsiSb.5 2019-11-25T10:25:21.939 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:21.936+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:22.032 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.029+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T10:25:22.193 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.191+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:22.216 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.214+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:22.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=3, truncated=0, next_marker='' 2019-11-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1086', time=2019-11-25T12:07:04.875957+0000, chain.objs.size()=7 2019-11-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~IJC-w9hep4-aw6w4Qn8f5vtIG2MUt6N.1 2019-11-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~IJC-w9hep4-aw6w4Qn8f5vtIG2MUt6N.1_1 2019-11-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~IJC-w9hep4-aw6w4Qn8f5vtIG2MUt6N.1_2 2019-11-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~IJC-w9hep4-aw6w4Qn8f5vtIG2MUt6N.2 2019-11-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~IJC-w9hep4-aw6w4Qn8f5vtIG2MUt6N.2_1 2019-11-25T10:25:22.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~IJC-w9hep4-aw6w4Qn8f5vtIG2MUt6N.2_2 2019-11-25T10:25:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~IJC-w9hep4-aw6w4Qn8f5vtIG2MUt6N.3 2019-11-25T10:25:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1135', time=2019-11-25T12:08:22.980694+0000, chain.objs.size()=9 2019-11-25T10:25:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__multipart_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.1 2019-11-25T10:25:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__shadow_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.1_1 2019-11-25T10:25:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__multipart_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.2 2019-11-25T10:25:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__shadow_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.2_1 2019-11-25T10:25:22.255 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.250+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__multipart_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.3 2019-11-25T10:25:22.462 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.460+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__shadow_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.3_1 2019-11-25T10:25:22.621 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.618+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__multipart_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.4 2019-11-25T10:25:22.651 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.649+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__shadow_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.4_1 2019-11-25T10:25:22.651 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.649+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.71__multipart_key1.2~AP4TUOxucCWmksLt_Jn1KPZdAaGi-8d.5 2019-11-25T10:25:22.652 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.649+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1241', time=2019-11-25T12:12:25.033412+0000, chain.objs.size()=7 2019-11-25T10:25:22.652 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.649+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~M3lOaV6zEOO4EGSWWVWW-4HrgzYpKIZ.1 2019-11-25T10:25:22.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.788+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~M3lOaV6zEOO4EGSWWVWW-4HrgzYpKIZ.1_1 2019-11-25T10:25:22.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.788+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~M3lOaV6zEOO4EGSWWVWW-4HrgzYpKIZ.1_2 2019-11-25T10:25:22.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.788+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~M3lOaV6zEOO4EGSWWVWW-4HrgzYpKIZ.2 2019-11-25T10:25:22.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.788+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~M3lOaV6zEOO4EGSWWVWW-4HrgzYpKIZ.2_1 2019-11-25T10:25:22.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.788+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~M3lOaV6zEOO4EGSWWVWW-4HrgzYpKIZ.2_2 2019-11-25T10:25:22.791 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:22.788+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~M3lOaV6zEOO4EGSWWVWW-4HrgzYpKIZ.3 2019-11-25T10:25:23.364 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.361+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:23.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.561+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T10:25:23.605 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.603+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:23.621 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.618+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:23.648 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:23.649 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='2~mpyi751_ACNJZowZVbyQwsjnru-xjgD', time=2019-11-25T12:14:11.873719+0000, chain.objs.size()=45 2019-11-25T10:25:23.649 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.1 2019-11-25T10:25:23.649 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.2 2019-11-25T10:25:23.649 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.3 2019-11-25T10:25:23.649 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.4 2019-11-25T10:25:23.649 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.5 2019-11-25T10:25:23.649 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.6 2019-11-25T10:25:23.650 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.7 2019-11-25T10:25:23.650 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.8 2019-11-25T10:25:23.650 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.9 2019-11-25T10:25:23.650 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.10 2019-11-25T10:25:23.650 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.11 2019-11-25T10:25:23.650 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.645+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.12 2019-11-25T10:25:23.904 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:23.902+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.13 2019-11-25T10:25:24.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.034+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.AUmy3o_aHecTbDI08huSUzCjRSE6RrN.14 2019-11-25T10:25:24.166 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.Pex-5tiZK07RM-RqhAo2c9sO5UkpWNm.15 2019-11-25T10:25:24.166 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.j5YTZ4hkLrKz7yl5dxcKFQSa8d5L9rZ.16 2019-11-25T10:25:24.166 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.GXLdOuYqnc8yxzR5lpIyQCEKt7kklvO.17 2019-11-25T10:25:24.166 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.mgESm27yuEIWZ6EAX7Rf3IO3E2LT65H.18 2019-11-25T10:25:24.166 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.6kerX8P41-V7M989s3RUdfOsPt-oiNp.19 2019-11-25T10:25:24.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.1pBHPTywUscSXnLJ3aSXVLJ_x2Zlqz1.20 2019-11-25T10:25:24.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.21 2019-11-25T10:25:24.259 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.256+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.22 2019-11-25T10:25:24.260 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.256+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.23 2019-11-25T10:25:24.260 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.256+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.24 2019-11-25T10:25:25.001 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.25 2019-11-25T10:25:25.001 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.26 2019-11-25T10:25:25.001 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.27 2019-11-25T10:25:25.001 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.28 2019-11-25T10:25:25.001 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.29 2019-11-25T10:25:25.002 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.30 2019-11-25T10:25:25.002 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.31 2019-11-25T10:25:25.002 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.32 2019-11-25T10:25:25.002 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.33 2019-11-25T10:25:25.002 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:24.998+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.34 2019-11-25T10:25:25.851 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:25.848+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.35 2019-11-25T10:25:26.153 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.151+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.36 2019-11-25T10:25:26.154 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.151+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.37 2019-11-25T10:25:26.154 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.151+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.38 2019-11-25T10:25:26.154 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.151+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.39 2019-11-25T10:25:26.187 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.184+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.40 2019-11-25T10:25:26.187 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.184+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.41 2019-11-25T10:25:26.187 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.184+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.42 2019-11-25T10:25:26.187 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.184+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.43 2019-11-25T10:25:26.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.184+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.44 2019-11-25T10:25:26.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.184+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.78__multipart_key1.2~mpyi751_ACNJZowZVbyQwsjnru-xjgD.45 2019-11-25T10:25:26.743 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:26.740+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:27.067 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.065+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T10:25:27.106 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.104+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:27.121 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.119+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:27.129 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.127+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:27.157 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.155+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T10:25:27.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.193+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:27.212 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.210+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:27.220 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.218+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:27.249 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.247+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T10:25:27.291 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.289+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:27.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.309+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:27.321 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.318+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:27.354 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.351+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T10:25:27.401 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.399+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:27.419 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.416+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:27.440 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:27.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1080', time=2019-11-25T12:06:42.929763+0000, chain.objs.size()=9 2019-11-25T10:25:27.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.1 2019-11-25T10:25:27.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.1_1 2019-11-25T10:25:27.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.2 2019-11-25T10:25:27.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.2_1 2019-11-25T10:25:27.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.3 2019-11-25T10:25:27.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.3_1 2019-11-25T10:25:27.442 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.4 2019-11-25T10:25:27.442 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.437+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.4_1 2019-11-25T10:25:27.442 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.438+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~KPnmbPeOryl3mVH3SduyZFduCf1-Lk_.5 2019-11-25T10:25:27.841 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.838+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:27.965 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:27.962+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T10:25:28.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.149+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:28.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.165+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:28.182 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:28.182 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1092', time=2019-11-25T12:07:28.265321+0000, chain.objs.size()=7 2019-11-25T10:25:28.182 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~W2RkAV0wG6WyJRGoMnEkR36dOi1seqD.1 2019-11-25T10:25:28.182 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~W2RkAV0wG6WyJRGoMnEkR36dOi1seqD.1_1 2019-11-25T10:25:28.182 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~W2RkAV0wG6WyJRGoMnEkR36dOi1seqD.1_2 2019-11-25T10:25:28.183 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~W2RkAV0wG6WyJRGoMnEkR36dOi1seqD.2 2019-11-25T10:25:28.183 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~W2RkAV0wG6WyJRGoMnEkR36dOi1seqD.2_1 2019-11-25T10:25:28.183 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__shadow_key1.2~W2RkAV0wG6WyJRGoMnEkR36dOi1seqD.2_2 2019-11-25T10:25:28.183 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.179+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.67__multipart_key1.2~W2RkAV0wG6WyJRGoMnEkR36dOi1seqD.3 2019-11-25T10:25:28.467 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.465+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:28.587 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.584+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T10:25:28.628 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.625+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:28.646 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.644+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:28.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T10:25:28.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1043', time=2019-11-25T12:03:40.949512+0000, chain.objs.size()=9 2019-11-25T10:25:28.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.1 2019-11-25T10:25:28.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__shadow_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.1_1 2019-11-25T10:25:28.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.2 2019-11-25T10:25:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__shadow_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.2_1 2019-11-25T10:25:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.3 2019-11-25T10:25:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__shadow_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.3_1 2019-11-25T10:25:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.4 2019-11-25T10:25:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__shadow_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.4_1 2019-11-25T10:25:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.5 2019-11-25T10:25:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1261', time=2019-11-25T12:13:12.128651+0000, chain.objs.size()=9 2019-11-25T10:25:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__multipart_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.1 2019-11-25T10:25:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__shadow_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.1_1 2019-11-25T10:25:28.667 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__multipart_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.2 2019-11-25T10:25:28.764 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.761+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__shadow_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.2_1 2019-11-25T10:25:28.772 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.770+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__multipart_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.3 2019-11-25T10:25:28.991 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.989+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__shadow_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.3_1 2019-11-25T10:25:28.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.989+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__multipart_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.4 2019-11-25T10:25:28.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.989+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__shadow_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.4_1 2019-11-25T10:25:28.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:28.989+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.76__multipart_key1.2~iOG1ssEChoSYd1lKpcwSh82QmF_EYPZ.5 2019-11-25T10:25:29.315 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.312+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:29.392 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.390+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T10:25:29.435 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.432+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:29.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.449+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:29.460 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.457+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:29.490 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.487+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T10:25:29.530 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.527+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:29.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.545+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:29.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1224', time=2019-11-25T12:11:17.630191+0000, chain.objs.size()=9 2019-11-25T10:25:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.1 2019-11-25T10:25:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.1_1 2019-11-25T10:25:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.2 2019-11-25T10:25:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.2_1 2019-11-25T10:25:29.568 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.3 2019-11-25T10:25:29.569 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.3_1 2019-11-25T10:25:29.569 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.4 2019-11-25T10:25:29.569 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.4_1 2019-11-25T10:25:29.569 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:29.564+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~EEcYSQpvQCymdMvNmwx9gv15Tyb77Gm.5 2019-11-25T10:25:30.049 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.047+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:30.114 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.112+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T10:25:30.162 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.160+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:30.178 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.175+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=2, truncated=0, next_marker='' 2019-11-25T10:25:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1185', time=2019-11-25T12:09:08.991599+0000, chain.objs.size()=9 2019-11-25T10:25:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.1 2019-11-25T10:25:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__shadow_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.1_1 2019-11-25T10:25:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.2 2019-11-25T10:25:30.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__shadow_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.2_1 2019-11-25T10:25:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.3 2019-11-25T10:25:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__shadow_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.3_1 2019-11-25T10:25:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.4 2019-11-25T10:25:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__shadow_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.4_1 2019-11-25T10:25:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.73__multipart_file.mpg.2~gCKw_U0ioIG5AVmRt6r3rYLghmlU9lk.5 2019-11-25T10:25:30.195 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1236', time=2019-11-25T12:12:06.661094+0000, chain.objs.size()=7 2019-11-25T10:25:30.196 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~PbWpHdabkmJw_dOHhNVMHpuB3QWsdq3.1 2019-11-25T10:25:30.196 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~PbWpHdabkmJw_dOHhNVMHpuB3QWsdq3.1_1 2019-11-25T10:25:30.196 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.191+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~PbWpHdabkmJw_dOHhNVMHpuB3QWsdq3.1_2 2019-11-25T10:25:30.438 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.435+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~PbWpHdabkmJw_dOHhNVMHpuB3QWsdq3.2 2019-11-25T10:25:30.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.573+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~PbWpHdabkmJw_dOHhNVMHpuB3QWsdq3.2_1 2019-11-25T10:25:30.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.573+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~PbWpHdabkmJw_dOHhNVMHpuB3QWsdq3.2_2 2019-11-25T10:25:30.664 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.661+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~PbWpHdabkmJw_dOHhNVMHpuB3QWsdq3.3 2019-11-25T10:25:30.980 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:30.977+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:31.107 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.103+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T10:25:31.207 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.204+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:31.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.230+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:31.252 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:31.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1246', time=2019-11-25T12:12:28.882880+0000, chain.objs.size()=7 2019-11-25T10:25:31.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~KsOR6CNm-_6_rU6idp86XrbXHyMWO5V.1 2019-11-25T10:25:31.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~KsOR6CNm-_6_rU6idp86XrbXHyMWO5V.1_1 2019-11-25T10:25:31.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~KsOR6CNm-_6_rU6idp86XrbXHyMWO5V.1_2 2019-11-25T10:25:31.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~KsOR6CNm-_6_rU6idp86XrbXHyMWO5V.2 2019-11-25T10:25:31.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~KsOR6CNm-_6_rU6idp86XrbXHyMWO5V.2_1 2019-11-25T10:25:31.253 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__shadow_key1.2~KsOR6CNm-_6_rU6idp86XrbXHyMWO5V.2_2 2019-11-25T10:25:31.254 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.249+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.75__multipart_key1.2~KsOR6CNm-_6_rU6idp86XrbXHyMWO5V.3 2019-11-25T10:25:31.734 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.731+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:31.975 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:31.973+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T10:25:32.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.091+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:32.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.134+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:32.166 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=1, truncated=0, next_marker='' 2019-11-25T10:25:32.166 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 20 garbage collection: RGWGC::process iterating over entry tag='103c5a94-eb69-4977-8c90-9f6d634d8761.4500.1041', time=2019-11-25T12:03:34.674443+0000, chain.objs.size()=9 2019-11-25T10:25:32.166 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.1 2019-11-25T10:25:32.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__shadow_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.1_1 2019-11-25T10:25:32.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.2 2019-11-25T10:25:32.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__shadow_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.2_1 2019-11-25T10:25:32.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.3 2019-11-25T10:25:32.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__shadow_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.3_1 2019-11-25T10:25:32.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.4 2019-11-25T10:25:32.167 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__shadow_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.4_1 2019-11-25T10:25:32.168 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.163+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing default.rgw.buckets.data:103c5a94-eb69-4977-8c90-9f6d634d8761.4358.64__multipart_key1.2~tNtf1mfiQb8vCLdjzwxEp25sWYc_EQm.5 2019-11-25T10:25:32.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.613+0000 7f41390f9ec0 5 garbage collection: RGWGC::process removing entries, marker: 2019-11-25T10:25:32.990 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:32.987+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T10:25:33.096 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.091+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:33.127 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.125+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:33.140 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.137+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:33.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.186+0000 7f41390f9ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T10:25:33.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.236+0000 7f40f6fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:25:33.270 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.267+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:33.305 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.302+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:33.327 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.324+0000 7f41390f9ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.420+0000 7f41390f9ec0 20 remove_watcher() i=0 2019-11-25T10:25:33.423 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.420+0000 7f41390f9ec0 2 removed watcher, disabling cache 2019-11-25T10:25:33.501 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.498+0000 7f41390f9ec0 20 remove_watcher() i=1 2019-11-25T10:25:33.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.564+0000 7f41390f9ec0 20 remove_watcher() i=2 2019-11-25T10:25:33.608 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.605+0000 7f41390f9ec0 20 remove_watcher() i=3 2019-11-25T10:25:33.647 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.644+0000 7f41390f9ec0 20 remove_watcher() i=4 2019-11-25T10:25:33.692 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.689+0000 7f41390f9ec0 20 remove_watcher() i=5 2019-11-25T10:25:33.729 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.726+0000 7f41390f9ec0 20 remove_watcher() i=6 2019-11-25T10:25:33.798 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.795+0000 7f41390f9ec0 20 remove_watcher() i=7 2019-11-25T10:25:33.805 INFO:teuthology.orchestra.run.smithi153:> radosgw-admin gc process --include-all 2019-11-25T10:25:33.915 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.911+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:33.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.939+0000 7f796ad83ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:33.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.939+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:33.951 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.948+0000 7f796ad83ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:33.951 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.948+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:33.959 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.957+0000 7f796ad83ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:33.959 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.957+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:33.979 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:33.976+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:34.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.147+0000 7f796ad83ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T10:25:34.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.147+0000 7f796ad83ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T10:25:34.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.147+0000 7f796ad83ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T10:25:34.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.147+0000 7f796ad83ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T10:25:34.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.148+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.161 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.158+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:34.161 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.158+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.167+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:34.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.167+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.182 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.180+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:34.183 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.180+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.199 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.196+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:34.199 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.196+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.211+0000 7f796ad83ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:34.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.211+0000 7f796ad83ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T10:25:34.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.211+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.222 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.220+0000 7f796ad83ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:34.222 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.220+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.232 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.230+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:34.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.230+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.249 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.246+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:34.249 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.246+0000 7f796ad83ec0 20 zonegroup default 2019-11-25T10:25:34.249 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.246+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.258 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.255+0000 7f796ad83ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:34.258 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.255+0000 7f796ad83ec0 10 Cannot find current period zone using local zone 2019-11-25T10:25:34.258 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.255+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.266 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.264+0000 7f796ad83ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:34.267 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.264+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.279 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.277+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:34.279 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.277+0000 7f796ad83ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:34.296 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.293+0000 7f796ad83ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:34.296 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.293+0000 7f796ad83ec0 20 zone default found 2019-11-25T10:25:34.296 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.293+0000 7f796ad83ec0 20 started sync module instance, tier type = 2019-11-25T10:25:34.297 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.293+0000 7f796ad83ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T10:25:34.669 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.666+0000 7f796ad83ec0 20 add_watcher() i=0 2019-11-25T10:25:34.669 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.666+0000 7f796ad83ec0 20 add_watcher() i=1 2019-11-25T10:25:34.669 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.666+0000 7f796ad83ec0 20 add_watcher() i=2 2019-11-25T10:25:34.669 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.666+0000 7f796ad83ec0 20 add_watcher() i=3 2019-11-25T10:25:34.670 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.666+0000 7f796ad83ec0 20 add_watcher() i=4 2019-11-25T10:25:34.670 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.666+0000 7f796ad83ec0 20 add_watcher() i=5 2019-11-25T10:25:34.670 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.666+0000 7f796ad83ec0 20 add_watcher() i=6 2019-11-25T10:25:34.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.698+0000 7f796ad83ec0 20 add_watcher() i=7 2019-11-25T10:25:34.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.698+0000 7f796ad83ec0 2 all 8 watchers are set, enabling cache 2019-11-25T10:25:34.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.698+0000 7f78abfff700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:25:34.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.710+0000 7f79587b2700 0 -- 172.21.15.153:0/137489312 >> v2:172.21.15.81:3300/0 conn(0x55a3f47e1470 msgr2=0x55a3f47f57c0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T10:25:34.751 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:34.748+0000 7f7957fb1700 0 -- 172.21.15.153:0/2714641865 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7f7898017590 msgr2=0x7f7898019aa0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T10:25:36.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.400+0000 7f78967fc700 20 reqs_thread_entry: start 2019-11-25T10:25:36.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.401+0000 7f788ffff700 20 reqs_thread_entry: start 2019-11-25T10:25:36.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.401+0000 7f796ad83ec0 20 init_complete bucket index max shards: 0 2019-11-25T10:25:36.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.402+0000 7f788e7fc700 20 reqs_thread_entry: start 2019-11-25T10:25:36.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.402+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=19, max_secs=3600, expired_only=0 2019-11-25T10:25:36.467 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.465+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:36.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.486+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:36.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.500+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:36.542 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.540+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=20, max_secs=3600, expired_only=0 2019-11-25T10:25:36.596 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.594+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:36.622 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.620+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:36.632 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.630+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:36.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.662+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=21, max_secs=3600, expired_only=0 2019-11-25T10:25:36.718 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.715+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:36.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.737+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:36.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.757+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:36.815 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.813+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=22, max_secs=3600, expired_only=0 2019-11-25T10:25:36.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.889+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:36.917 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.915+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:36.931 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.928+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:36.965 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:36.962+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=23, max_secs=3600, expired_only=0 2019-11-25T10:25:37.056 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.053+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.094 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.091+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:37.112 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.109+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.175 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.172+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=24, max_secs=3600, expired_only=0 2019-11-25T10:25:37.264 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.261+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.296+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:37.350 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.314+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.368+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=25, max_secs=3600, expired_only=0 2019-11-25T10:25:37.439 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.436+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.486 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.484+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:37.500 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.498+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.542+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=26, max_secs=3600, expired_only=0 2019-11-25T10:25:37.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.583+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.601 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.599+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:37.610 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.608+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.640 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.637+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=27, max_secs=3600, expired_only=0 2019-11-25T10:25:37.679 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.676+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.694 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.692+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:37.707 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.705+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.769 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.766+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=28, max_secs=3600, expired_only=0 2019-11-25T10:25:37.818 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.816+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.838 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.836+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:37.848 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.846+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.892 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.890+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=29, max_secs=3600, expired_only=0 2019-11-25T10:25:37.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.942+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:37.960 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.957+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:37.968 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:37.966+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.009 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.006+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=30, max_secs=3600, expired_only=0 2019-11-25T10:25:38.077 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.074+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.113 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.110+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:38.122 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.120+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.180 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.177+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=31, max_secs=3600, expired_only=0 2019-11-25T10:25:38.265 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.263+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.304+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.326+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.393 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.390+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=0, max_secs=3600, expired_only=0 2019-11-25T10:25:38.461 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.459+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.481+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:38.501 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.498+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.544+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=1, max_secs=3600, expired_only=0 2019-11-25T10:25:38.604 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.601+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.628 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.625+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:38.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.637+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.677 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.675+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=2, max_secs=3600, expired_only=0 2019-11-25T10:25:38.721 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.719+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.737 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.734+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:38.748 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.746+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.786 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.783+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=3, max_secs=3600, expired_only=0 2019-11-25T10:25:38.850 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.847+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.866 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.864+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:38.875 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.872+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:38.914 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.911+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=4, max_secs=3600, expired_only=0 2019-11-25T10:25:38.977 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:38.975+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.010 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.007+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:39.027 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.024+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.073 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.070+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=5, max_secs=3600, expired_only=0 2019-11-25T10:25:39.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.147+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.171+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:39.199 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.197+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.258 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.256+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=6, max_secs=3600, expired_only=0 2019-11-25T10:25:39.313 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.310+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.333 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.331+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:39.347 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.344+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.387 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.384+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=7, max_secs=3600, expired_only=0 2019-11-25T10:25:39.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.462+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.486+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:39.501 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.499+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.558 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.555+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=8, max_secs=3600, expired_only=0 2019-11-25T10:25:39.644 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.641+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.670 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.668+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:39.686 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.683+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.730+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=9, max_secs=3600, expired_only=0 2019-11-25T10:25:39.808 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.805+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.838 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.835+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:39.855 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.853+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:39.902 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.899+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=10, max_secs=3600, expired_only=0 2019-11-25T10:25:39.977 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:39.974+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.017 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.014+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:40.035 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.033+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.105 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.102+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=11, max_secs=3600, expired_only=0 2019-11-25T10:25:40.207 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.205+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.233 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.231+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:40.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.244+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.295 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.292+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=12, max_secs=3600, expired_only=0 2019-11-25T10:25:40.366 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.363+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.400+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:40.423 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.420+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.469 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.466+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=13, max_secs=3600, expired_only=0 2019-11-25T10:25:40.560 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.558+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.598 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.596+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:40.611 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.608+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.645 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.642+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=14, max_secs=3600, expired_only=0 2019-11-25T10:25:40.689 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.687+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.711 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.708+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:40.720 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.718+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.763 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.760+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=15, max_secs=3600, expired_only=0 2019-11-25T10:25:40.822 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.820+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.845 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.843+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:40.856 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.853+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:40.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.918+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=16, max_secs=3600, expired_only=0 2019-11-25T10:25:40.987 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:40.984+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:41.025 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.022+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:41.048 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.045+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:41.096 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.094+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=17, max_secs=3600, expired_only=0 2019-11-25T10:25:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.167+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:41.221 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.218+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:41.236 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.233+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:41.315 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.312+0000 7f796ad83ec0 20 garbage collection: RGWGC::process entered with GC index_shard=18, max_secs=3600, expired_only=0 2019-11-25T10:25:41.380 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.378+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned with returned:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:41.405 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.403+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_list returned ENOENT for non expired entries, so setting cache entry to TRUE 2019-11-25T10:25:41.414 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.412+0000 7f796ad83ec0 20 garbage collection: RGWGC::process cls_rgw_gc_queue_list_entries returned with return value:0, entries.size=0, truncated=0, next_marker='' 2019-11-25T10:25:41.491 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.489+0000 7f796ad83ec0 20 remove_watcher() i=0 2019-11-25T10:25:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.489+0000 7f796ad83ec0 2 removed watcher, disabling cache 2019-11-25T10:25:41.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.550+0000 7f796ad83ec0 20 remove_watcher() i=1 2019-11-25T10:25:41.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.606+0000 7f796ad83ec0 20 remove_watcher() i=2 2019-11-25T10:25:41.672 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.669+0000 7f796ad83ec0 20 remove_watcher() i=3 2019-11-25T10:25:41.720 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.718+0000 7f796ad83ec0 20 remove_watcher() i=4 2019-11-25T10:25:41.772 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.769+0000 7f796ad83ec0 20 remove_watcher() i=5 2019-11-25T10:25:41.835 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.832+0000 7f796ad83ec0 20 remove_watcher() i=6 2019-11-25T10:25:41.875 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:41.872+0000 7f796ad83ec0 20 remove_watcher() i=7 2019-11-25T10:25:41.883 INFO:teuthology.run_tasks:Running task s3tests... 2019-11-25T10:25:41.934 DEBUG:tasks.s3tests:s3tests config is {'client.0': {'rgw_server': 'client.0', 'force-branch': 'ceph-master', 'slow_backend': True}} 2019-11-25T10:25:41.934 INFO:tasks.s3tests:Downloading s3-tests... 2019-11-25T10:25:41.934 INFO:tasks.s3tests:Using branch 'ceph-master' for s3tests 2019-11-25T10:25:41.935 INFO:teuthology.orchestra.run.smithi153:> git clone -b ceph-master git://git.ceph.com/git/s3-tests.git /home/ubuntu/cephtest/s3-tests 2019-11-25T10:25:42.040 INFO:teuthology.orchestra.run.smithi153.stdout:Cloning into '/home/ubuntu/cephtest/s3-tests'... 2019-11-25T10:25:42.465 INFO:tasks.s3tests:Creating rgw users... 2019-11-25T10:25:42.466 DEBUG:tasks.s3tests:Creating user bar.client.0 on client.0 2019-11-25T10:25:42.466 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid bar.client.0 --display-name 'Mr. bar.client.0' --access-key JBHEYAKBJIAFTAJDWIHI --secret 5rjN1VjsBGVoo8bR7o+wdhKS2xUwsUAc2tj2AcdUEoVq2Tf40U3J3Q== --email bar.client.0+test@test.test --cluster ceph 2019-11-25T10:25:42.610 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.608+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.643 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.640+0000 7fc728fdbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:42.643 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.640+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.651 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.648+0000 7fc728fdbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:42.651 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.648+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.659+0000 7fc728fdbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:42.661 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.659+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.681 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.679+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:42.907 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.905+0000 7fc728fdbec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T10:25:42.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.905+0000 7fc728fdbec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T10:25:42.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.905+0000 7fc728fdbec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T10:25:42.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.905+0000 7fc728fdbec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T10:25:42.908 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.905+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.919 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.917+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:42.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.917+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.927+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:42.929 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.927+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.943 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.940+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:42.943 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.940+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.956 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.953+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:42.956 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.953+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.963 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.960+0000 7fc728fdbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:42.963 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.960+0000 7fc728fdbec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T10:25:42.963 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.960+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.968 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.966+0000 7fc728fdbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:42.968 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.966+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.977 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.975+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:42.977 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.975+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.986 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.983+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:42.986 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.983+0000 7fc728fdbec0 20 zonegroup default 2019-11-25T10:25:42.986 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.983+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.992 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.990+0000 7fc728fdbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:42.993 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.990+0000 7fc728fdbec0 10 Cannot find current period zone using local zone 2019-11-25T10:25:42.993 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.990+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:42.999 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.997+0000 7fc728fdbec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:42.999 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:42.997+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:43.009 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.006+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:43.009 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.006+0000 7fc728fdbec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:43.018 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.016+0000 7fc728fdbec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.016+0000 7fc728fdbec0 20 zone default found 2019-11-25T10:25:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.016+0000 7fc728fdbec0 20 started sync module instance, tier type = 2019-11-25T10:25:43.019 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.016+0000 7fc728fdbec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.306+0000 7fc728fdbec0 20 add_watcher() i=0 2019-11-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.306+0000 7fc728fdbec0 20 add_watcher() i=1 2019-11-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.306+0000 7fc728fdbec0 20 add_watcher() i=2 2019-11-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.306+0000 7fc728fdbec0 20 add_watcher() i=3 2019-11-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.306+0000 7fc728fdbec0 20 add_watcher() i=4 2019-11-25T10:25:43.309 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.306+0000 7fc728fdbec0 20 add_watcher() i=5 2019-11-25T10:25:43.310 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.306+0000 7fc728fdbec0 20 add_watcher() i=6 2019-11-25T10:25:43.337 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.335+0000 7fc728fdbec0 20 add_watcher() i=7 2019-11-25T10:25:43.337 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.335+0000 7fc728fdbec0 2 all 8 watchers are set, enabling cache 2019-11-25T10:25:43.338 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.335+0000 7fc6e6fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:25:43.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.337+0000 7fc716209700 0 -- 172.21.15.153:0/643516021 >> v2:172.21.15.153:3300/0 conn(0x563b82b3e0a0 msgr2=0x563b82b44980 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T10:25:43.381 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:43.379+0000 7fc715a08700 0 -- 172.21.15.153:0/3355446602 >> [v2:172.21.15.81:6800/9718,v1:172.21.15.81:6801/9718] conn(0x563b82b56210 msgr2=0x563b82b58630 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T10:25:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.159+0000 7fc6557fa700 20 reqs_thread_entry: start 2019-11-25T10:25:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.159+0000 7fc64effd700 20 reqs_thread_entry: start 2019-11-25T10:25:45.162 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.159+0000 7fc728fdbec0 20 init_complete bucket index max shards: 0 2019-11-25T10:25:45.163 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.160+0000 7fc64d7fa700 20 reqs_thread_entry: start 2019-11-25T10:25:45.163 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.160+0000 7fc728fdbec0 20 get_system_obj_state: rctx=0x7ffc757ac118 obj=default.rgw.meta:users.uid:bar.client.0 state=0x563b82bbcd70 s->prefetch_data=0 2019-11-25T10:25:45.163 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.160+0000 7fc728fdbec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-11-25T10:25:45.172 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.169+0000 7fc728fdbec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-11-25T10:25:45.172 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.170+0000 7fc728fdbec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T10:25:45.173 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.170+0000 7fc728fdbec0 20 get_system_obj_state: rctx=0x7ffc757ac128 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x563b82bbeb00 s->prefetch_data=0 2019-11-25T10:25:45.173 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.170+0000 7fc728fdbec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-11-25T10:25:45.178 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.175+0000 7fc728fdbec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-11-25T10:25:45.178 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.175+0000 7fc728fdbec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-25T10:25:45.178 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.175+0000 7fc728fdbec0 20 get_system_obj_state: rctx=0x7ffc757ac128 obj=default.rgw.meta:users.keys:JBHEYAKBJIAFTAJDWIHI state=0x563b82bbeb00 s->prefetch_data=0 2019-11-25T10:25:45.178 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.175+0000 7fc728fdbec0 10 cache get: name=default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI : miss 2019-11-25T10:25:45.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.183+0000 7fc728fdbec0 10 cache put: name=default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI info.flags=0x0 2019-11-25T10:25:45.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.183+0000 7fc728fdbec0 10 adding default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI to cache LRU end 2019-11-25T10:25:45.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.183+0000 7fc728fdbec0 20 get_system_obj_state: rctx=0x7ffc757abd38 obj=default.rgw.meta:users.keys:JBHEYAKBJIAFTAJDWIHI state=0x563b82bbeb00 s->prefetch_data=0 2019-11-25T10:25:45.187 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.183+0000 7fc728fdbec0 10 cache get: name=default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI : type miss (requested=0x6, cached=0x0) 2019-11-25T10:25:45.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.191+0000 7fc728fdbec0 10 cache put: name=default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI info.flags=0x0 2019-11-25T10:25:45.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.191+0000 7fc728fdbec0 10 moving default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI to cache LRU end 2019-11-25T10:25:45.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.191+0000 7fc728fdbec0 20 get_system_obj_state: rctx=0x7ffc757abeb8 obj=default.rgw.meta:users.keys:JBHEYAKBJIAFTAJDWIHI state=0x563b82bbeb00 s->prefetch_data=0 2019-11-25T10:25:45.194 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.191+0000 7fc728fdbec0 10 cache get: name=default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI : type miss (requested=0x6, cached=0x0) 2019-11-25T10:25:45.201 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.198+0000 7fc728fdbec0 10 cache put: name=default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI info.flags=0x0 2019-11-25T10:25:45.201 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.198+0000 7fc728fdbec0 10 moving default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI to cache LRU end 2019-11-25T10:25:45.271 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.268+0000 7fc728fdbec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-25T10:25:45.271 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.268+0000 7fc728fdbec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T10:25:45.271 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.268+0000 7fc728fdbec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=617 2019-11-25T10:25:45.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.284+0000 7fc6f77fe700 10 RGWWatcher::handle_notify() notify_id 579820585025 cookie 94813595821104 notifier 4622 bl.length()=617 2019-11-25T10:25:45.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.284+0000 7fc6f77fe700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-25T10:25:45.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.284+0000 7fc6f77fe700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T10:25:45.565 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.562+0000 7fc728fdbec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-11-25T10:25:45.565 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.562+0000 7fc728fdbec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-25T10:25:45.565 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.562+0000 7fc728fdbec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-11-25T10:25:45.575 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.573+0000 7fc6f77fe700 10 RGWWatcher::handle_notify() notify_id 579820585026 cookie 94813595821104 notifier 4622 bl.length()=194 2019-11-25T10:25:45.575 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.573+0000 7fc6f77fe700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-11-25T10:25:45.576 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.573+0000 7fc6f77fe700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-25T10:25:45.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.951+0000 7fc728fdbec0 10 cache put: name=default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI info.flags=0x7 2019-11-25T10:25:45.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.951+0000 7fc728fdbec0 10 moving default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI to cache LRU end 2019-11-25T10:25:45.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.951+0000 7fc728fdbec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=186 2019-11-25T10:25:45.975 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.972+0000 7fc6f77fe700 10 RGWWatcher::handle_notify() notify_id 579820585019 cookie 94813595817744 notifier 4622 bl.length()=186 2019-11-25T10:25:45.975 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.972+0000 7fc6f77fe700 10 cache put: name=default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI info.flags=0x7 2019-11-25T10:25:45.975 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:45.972+0000 7fc6f77fe700 10 moving default.rgw.meta+users.keys+JBHEYAKBJIAFTAJDWIHI to cache LRU end 2019-11-25T10:25:46.162 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2019-11-25T10:25:46.162 INFO:teuthology.orchestra.run.smithi153.stdout: "user_id": "bar.client.0", 2019-11-25T10:25:46.162 INFO:teuthology.orchestra.run.smithi153.stdout: "display_name": "Mr. bar.client.0", 2019-11-25T10:25:46.162 INFO:teuthology.orchestra.run.smithi153.stdout: "email": "bar.client.0+test@test.test", 2019-11-25T10:25:46.162 INFO:teuthology.orchestra.run.smithi153.stdout: "suspended": 0, 2019-11-25T10:25:46.162 INFO:teuthology.orchestra.run.smithi153.stdout: "max_buckets": 1000, 2019-11-25T10:25:46.163 INFO:teuthology.orchestra.run.smithi153.stdout: "subusers": [], 2019-11-25T10:25:46.163 INFO:teuthology.orchestra.run.smithi153.stdout: "keys": [ 2019-11-25T10:25:46.163 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T10:25:46.163 INFO:teuthology.orchestra.run.smithi153.stdout: "user": "bar.client.0", 2019-11-25T10:25:46.163 INFO:teuthology.orchestra.run.smithi153.stdout: "access_key": "JBHEYAKBJIAFTAJDWIHI", 2019-11-25T10:25:46.163 INFO:teuthology.orchestra.run.smithi153.stdout: "secret_key": "5rjN1VjsBGVoo8bR7o+wdhKS2xUwsUAc2tj2AcdUEoVq2Tf40U3J3Q==" 2019-11-25T10:25:46.163 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T10:25:46.164 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T10:25:46.164 INFO:teuthology.orchestra.run.smithi153.stdout: "swift_keys": [], 2019-11-25T10:25:46.164 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": [], 2019-11-25T10:25:46.164 INFO:teuthology.orchestra.run.smithi153.stdout: "op_mask": "read, write, delete", 2019-11-25T10:25:46.164 INFO:teuthology.orchestra.run.smithi153.stdout: "default_placement": "", 2019-11-25T10:25:46.164 INFO:teuthology.orchestra.run.smithi153.stdout: "default_storage_class": "", 2019-11-25T10:25:46.164 INFO:teuthology.orchestra.run.smithi153.stdout: "placement_tags": [], 2019-11-25T10:25:46.165 INFO:teuthology.orchestra.run.smithi153.stdout: "bucket_quota": { 2019-11-25T10:25:46.165 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T10:25:46.165 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T10:25:46.165 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T10:25:46.165 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T10:25:46.165 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T10:25:46.165 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T10:25:46.166 INFO:teuthology.orchestra.run.smithi153.stdout: "user_quota": { 2019-11-25T10:25:46.166 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T10:25:46.166 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T10:25:46.166 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T10:25:46.166 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T10:25:46.166 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T10:25:46.166 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T10:25:46.166 INFO:teuthology.orchestra.run.smithi153.stdout: "temp_url_keys": [], 2019-11-25T10:25:46.167 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "rgw", 2019-11-25T10:25:46.167 INFO:teuthology.orchestra.run.smithi153.stdout: "mfa_ids": [] 2019-11-25T10:25:46.167 INFO:teuthology.orchestra.run.smithi153.stdout:} 2019-11-25T10:25:46.167 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:25:46.215 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.212+0000 7fc728fdbec0 20 remove_watcher() i=0 2019-11-25T10:25:46.215 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.212+0000 7fc728fdbec0 2 removed watcher, disabling cache 2019-11-25T10:25:46.281 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.278+0000 7fc728fdbec0 20 remove_watcher() i=1 2019-11-25T10:25:46.329 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.326+0000 7fc728fdbec0 20 remove_watcher() i=2 2019-11-25T10:25:46.381 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.379+0000 7fc728fdbec0 20 remove_watcher() i=3 2019-11-25T10:25:46.440 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.437+0000 7fc728fdbec0 20 remove_watcher() i=4 2019-11-25T10:25:46.483 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.481+0000 7fc728fdbec0 20 remove_watcher() i=5 2019-11-25T10:25:46.527 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.524+0000 7fc728fdbec0 20 remove_watcher() i=6 2019-11-25T10:25:46.560 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.558+0000 7fc728fdbec0 20 remove_watcher() i=7 2019-11-25T10:25:46.567 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid bar.client.0 --totp-serial 9497224990 --totp-seed QN3P5JQLXRFKPMX6YPXSKFJNIGNFKRIRON26IHU4WCHK5AQHXP7LS5XBECA4SSG7 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-11-25T10:25:46.718 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.715+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:46.766 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.763+0000 7f23037f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:46.766 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.763+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:46.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.776+0000 7f23037f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:46.778 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.776+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:46.787 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.785+0000 7f23037f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:46.787 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.785+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:46.804 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:46.802+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:47.012 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.009+0000 7f23037f9ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T10:25:47.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.009+0000 7f23037f9ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T10:25:47.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.009+0000 7f23037f9ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T10:25:47.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.009+0000 7f23037f9ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T10:25:47.013 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.010+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.028 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.025+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:47.028 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.025+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.044+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:47.046 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.044+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.058+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:47.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.058+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.075 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.072+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:47.075 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.072+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.085+0000 7f23037f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:47.087 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.085+0000 7f23037f9ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T10:25:47.088 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.085+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.095 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.092+0000 7f23037f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:47.095 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.092+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.107 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.104+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:47.107 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.104+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.126 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.123+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:47.126 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.123+0000 7f23037f9ec0 20 zonegroup default 2019-11-25T10:25:47.126 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.123+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.134 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.131+0000 7f23037f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:47.134 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.131+0000 7f23037f9ec0 10 Cannot find current period zone using local zone 2019-11-25T10:25:47.134 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.131+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.139+0000 7f23037f9ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:47.141 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.139+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.156+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:47.158 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.156+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:47.171 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.168+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:47.171 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.168+0000 7f23037f9ec0 20 zone default found 2019-11-25T10:25:47.171 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.168+0000 7f23037f9ec0 20 started sync module instance, tier type = 2019-11-25T10:25:47.171 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.168+0000 7f23037f9ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T10:25:47.519 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.517+0000 7f23037f9ec0 20 add_watcher() i=0 2019-11-25T10:25:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.517+0000 7f23037f9ec0 20 add_watcher() i=1 2019-11-25T10:25:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.517+0000 7f23037f9ec0 20 add_watcher() i=2 2019-11-25T10:25:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.517+0000 7f23037f9ec0 20 add_watcher() i=3 2019-11-25T10:25:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.517+0000 7f23037f9ec0 20 add_watcher() i=4 2019-11-25T10:25:47.520 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.517+0000 7f23037f9ec0 20 add_watcher() i=5 2019-11-25T10:25:47.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.518+0000 7f23037f9ec0 20 add_watcher() i=6 2019-11-25T10:25:47.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.550+0000 7f23037f9ec0 20 add_watcher() i=7 2019-11-25T10:25:47.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.550+0000 7f23037f9ec0 2 all 8 watchers are set, enabling cache 2019-11-25T10:25:47.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.550+0000 7f22c17e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:25:47.567 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.564+0000 7f22f0a27700 0 -- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x564d20ab1b50 msgr2=0x564d20ab6990 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T10:25:47.766 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:47.763+0000 7f22f0a27700 0 -- 172.21.15.153:0/3466284591 >> [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] conn(0x564d20ac11f0 msgr2=0x564d20acab30 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T10:25:49.561 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.559+0000 7f22337fe700 20 reqs_thread_entry: start 2019-11-25T10:25:49.562 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.559+0000 7f22317fa700 20 reqs_thread_entry: start 2019-11-25T10:25:49.562 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.559+0000 7f23037f9ec0 20 init_complete bucket index max shards: 0 2019-11-25T10:25:49.562 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.560+0000 7f221f7fe700 20 reqs_thread_entry: start 2019-11-25T10:25:49.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.560+0000 7f23037f9ec0 20 get_system_obj_state: rctx=0x7ffc562f49d8 obj=default.rgw.meta:users.uid:bar.client.0 state=0x564d20b2b950 s->prefetch_data=0 2019-11-25T10:25:49.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.560+0000 7f23037f9ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-11-25T10:25:49.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.576+0000 7f23037f9ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-11-25T10:25:49.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.576+0000 7f23037f9ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T10:25:49.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.576+0000 7f23037f9ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T10:25:49.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.576+0000 7f23037f9ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T10:25:49.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.576+0000 7f23037f9ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-25T10:25:49.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.576+0000 7f23037f9ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:49.591 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.589+0000 7f23037f9ec0 20 rados_obj.operate() r=0 bl.length=432 2019-11-25T10:25:49.592 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.589+0000 7f23037f9ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-11-25T10:25:49.592 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:49.589+0000 7f23037f9ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T10:25:51.895 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:51.893+0000 7f23037f9ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-25T10:25:51.896 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:51.893+0000 7f23037f9ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T10:25:51.896 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:51.893+0000 7f23037f9ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=631 2019-11-25T10:25:51.916 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:51.914+0000 7f22e67fc700 10 RGWWatcher::handle_notify() notify_id 588410519619 cookie 94889260463712 notifier 4867 bl.length()=631 2019-11-25T10:25:51.916 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:51.914+0000 7f22e67fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-25T10:25:51.917 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:51.914+0000 7f22e67fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T10:25:52.787 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:52.784+0000 7f23037f9ec0 20 remove_watcher() i=0 2019-11-25T10:25:52.787 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:52.784+0000 7f23037f9ec0 2 removed watcher, disabling cache 2019-11-25T10:25:52.860 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:52.857+0000 7f23037f9ec0 20 remove_watcher() i=1 2019-11-25T10:25:52.928 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:52.925+0000 7f23037f9ec0 20 remove_watcher() i=2 2019-11-25T10:25:52.986 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:52.984+0000 7f23037f9ec0 20 remove_watcher() i=3 2019-11-25T10:25:53.040 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.037+0000 7f23037f9ec0 20 remove_watcher() i=4 2019-11-25T10:25:53.096 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.094+0000 7f23037f9ec0 20 remove_watcher() i=5 2019-11-25T10:25:53.161 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.159+0000 7f23037f9ec0 20 remove_watcher() i=6 2019-11-25T10:25:53.199 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.196+0000 7f23037f9ec0 20 remove_watcher() i=7 2019-11-25T10:25:53.207 DEBUG:tasks.s3tests:Creating user foo.client.0 on client.0 2019-11-25T10:25:53.208 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid foo.client.0 --display-name 'Mr. foo.client.0' --access-key LBFZNLAOSMYTHLEKZWHV --secret Ag9AAs4UYPrDJ3/6tjUEOOFIkemtkueOJpL0jWk8YVz4XEzIJ0sklw== --email foo.client.0+test@test.test --cluster ceph 2019-11-25T10:25:53.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.336+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.368+0000 7fe25955fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:53.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.368+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.381 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.378+0000 7fe25955fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:53.381 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.378+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.395 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.393+0000 7fe25955fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:53.395 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.393+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.422 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.419+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:53.726 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.723+0000 7fe25955fec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T10:25:53.726 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.723+0000 7fe25955fec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T10:25:53.726 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.723+0000 7fe25955fec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T10:25:53.726 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.723+0000 7fe25955fec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T10:25:53.727 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.723+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.736+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:53.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.736+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.757 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.755+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:53.757 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.755+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.772+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:53.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.772+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.790+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:53.793 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.790+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.802 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.800+0000 7fe25955fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:53.803 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.800+0000 7fe25955fec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T10:25:53.803 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.800+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.812 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.809+0000 7fe25955fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:53.812 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.810+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.823+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:53.825 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.823+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.840 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.837+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:53.840 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.837+0000 7fe25955fec0 20 zonegroup default 2019-11-25T10:25:53.840 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.837+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.846+0000 7fe25955fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.846+0000 7fe25955fec0 10 Cannot find current period zone using local zone 2019-11-25T10:25:53.849 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.846+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.852+0000 7fe25955fec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:53.855 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.852+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.871 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.868+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:53.871 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.868+0000 7fe25955fec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:53.885 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.883+0000 7fe25955fec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.883+0000 7fe25955fec0 20 zone default found 2019-11-25T10:25:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.883+0000 7fe25955fec0 20 started sync module instance, tier type = 2019-11-25T10:25:53.886 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:53.883+0000 7fe25955fec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T10:25:54.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.300+0000 7fe25955fec0 20 add_watcher() i=0 2019-11-25T10:25:54.303 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.300+0000 7fe25955fec0 20 add_watcher() i=1 2019-11-25T10:25:54.304 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.300+0000 7fe25955fec0 20 add_watcher() i=2 2019-11-25T10:25:54.304 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.300+0000 7fe25955fec0 20 add_watcher() i=3 2019-11-25T10:25:54.304 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.300+0000 7fe25955fec0 20 add_watcher() i=4 2019-11-25T10:25:54.304 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.300+0000 7fe25955fec0 20 add_watcher() i=5 2019-11-25T10:25:54.308 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.305+0000 7fe25955fec0 20 add_watcher() i=6 2019-11-25T10:25:54.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.339+0000 7fe25955fec0 20 add_watcher() i=7 2019-11-25T10:25:54.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.339+0000 7fe25955fec0 2 all 8 watchers are set, enabling cache 2019-11-25T10:25:54.342 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.339+0000 7fe216fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:25:54.345 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.343+0000 7fe245f8c700 0 -- 172.21.15.153:0/3946761857 >> v2:172.21.15.81:3300/0 conn(0x555a939ab0d0 msgr2=0x555a939ad5a0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0)._try_send injecting socket failure 2019-11-25T10:25:54.382 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:54.379+0000 7fe245f8c700 0 -- 172.21.15.153:0/2210833213 >> [v2:172.21.15.81:6800/9718,v1:172.21.15.81:6801/9718] conn(0x555a939c3b00 msgr2=0x555a939c5f20 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T10:25:56.371 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.368+0000 7fe1857fa700 20 reqs_thread_entry: start 2019-11-25T10:25:56.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.369+0000 7fe17effd700 20 reqs_thread_entry: start 2019-11-25T10:25:56.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.369+0000 7fe25955fec0 20 init_complete bucket index max shards: 0 2019-11-25T10:25:56.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.369+0000 7fe17d7fa700 20 reqs_thread_entry: start 2019-11-25T10:25:56.373 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.369+0000 7fe25955fec0 20 get_system_obj_state: rctx=0x7ffd672d91c8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x555a93a2d2f0 s->prefetch_data=0 2019-11-25T10:25:56.373 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.369+0000 7fe25955fec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-11-25T10:25:56.379 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.376+0000 7fe25955fec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-11-25T10:25:56.379 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.376+0000 7fe25955fec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T10:25:56.379 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.376+0000 7fe25955fec0 20 get_system_obj_state: rctx=0x7ffd672d91d8 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x555a93a2f080 s->prefetch_data=0 2019-11-25T10:25:56.379 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.376+0000 7fe25955fec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-11-25T10:25:56.386 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.384+0000 7fe25955fec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-11-25T10:25:56.386 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.384+0000 7fe25955fec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-25T10:25:56.386 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.384+0000 7fe25955fec0 20 get_system_obj_state: rctx=0x7ffd672d91d8 obj=default.rgw.meta:users.keys:LBFZNLAOSMYTHLEKZWHV state=0x555a93a2f080 s->prefetch_data=0 2019-11-25T10:25:56.387 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.384+0000 7fe25955fec0 10 cache get: name=default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV : miss 2019-11-25T10:25:56.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.400+0000 7fe25955fec0 10 cache put: name=default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV info.flags=0x0 2019-11-25T10:25:56.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.400+0000 7fe25955fec0 10 adding default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV to cache LRU end 2019-11-25T10:25:56.402 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.400+0000 7fe25955fec0 20 get_system_obj_state: rctx=0x7ffd672d8de8 obj=default.rgw.meta:users.keys:LBFZNLAOSMYTHLEKZWHV state=0x555a93a2f080 s->prefetch_data=0 2019-11-25T10:25:56.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.400+0000 7fe25955fec0 10 cache get: name=default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV : type miss (requested=0x6, cached=0x0) 2019-11-25T10:25:56.409 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.407+0000 7fe25955fec0 10 cache put: name=default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV info.flags=0x0 2019-11-25T10:25:56.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.407+0000 7fe25955fec0 10 moving default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV to cache LRU end 2019-11-25T10:25:56.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.407+0000 7fe25955fec0 20 get_system_obj_state: rctx=0x7ffd672d8f68 obj=default.rgw.meta:users.keys:LBFZNLAOSMYTHLEKZWHV state=0x555a93a2f080 s->prefetch_data=0 2019-11-25T10:25:56.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.407+0000 7fe25955fec0 10 cache get: name=default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV : type miss (requested=0x6, cached=0x0) 2019-11-25T10:25:56.415 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.412+0000 7fe25955fec0 10 cache put: name=default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV info.flags=0x0 2019-11-25T10:25:56.415 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.412+0000 7fe25955fec0 10 moving default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV to cache LRU end 2019-11-25T10:25:56.451 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.448+0000 7fe25955fec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-25T10:25:56.451 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.448+0000 7fe25955fec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T10:25:56.451 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.448+0000 7fe25955fec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=617 2019-11-25T10:25:56.460 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.457+0000 7fe2277fe700 10 RGWWatcher::handle_notify() notify_id 592705486908 cookie 93847511714368 notifier 4637 bl.length()=617 2019-11-25T10:25:56.460 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.457+0000 7fe2277fe700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-25T10:25:56.460 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.457+0000 7fe2277fe700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T10:25:56.746 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.744+0000 7fe25955fec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-11-25T10:25:56.747 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.744+0000 7fe25955fec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-25T10:25:56.747 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.744+0000 7fe25955fec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-11-25T10:25:56.758 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.756+0000 7fe2277fe700 10 RGWWatcher::handle_notify() notify_id 592705486879 cookie 93847511730480 notifier 4637 bl.length()=194 2019-11-25T10:25:56.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.756+0000 7fe2277fe700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-11-25T10:25:56.759 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.756+0000 7fe2277fe700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-25T10:25:56.985 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.982+0000 7fe25955fec0 10 cache put: name=default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV info.flags=0x7 2019-11-25T10:25:56.985 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.982+0000 7fe25955fec0 10 moving default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV to cache LRU end 2019-11-25T10:25:56.985 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:56.982+0000 7fe25955fec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=186 2019-11-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.000+0000 7fe2277fe700 10 RGWWatcher::handle_notify() notify_id 592705486920 cookie 93847511724704 notifier 4637 bl.length()=186 2019-11-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.000+0000 7fe2277fe700 10 cache put: name=default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV info.flags=0x7 2019-11-25T10:25:57.003 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.000+0000 7fe2277fe700 10 moving default.rgw.meta+users.keys+LBFZNLAOSMYTHLEKZWHV to cache LRU end 2019-11-25T10:25:57.198 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2019-11-25T10:25:57.198 INFO:teuthology.orchestra.run.smithi153.stdout: "user_id": "foo.client.0", 2019-11-25T10:25:57.198 INFO:teuthology.orchestra.run.smithi153.stdout: "display_name": "Mr. foo.client.0", 2019-11-25T10:25:57.198 INFO:teuthology.orchestra.run.smithi153.stdout: "email": "foo.client.0+test@test.test", 2019-11-25T10:25:57.199 INFO:teuthology.orchestra.run.smithi153.stdout: "suspended": 0, 2019-11-25T10:25:57.199 INFO:teuthology.orchestra.run.smithi153.stdout: "max_buckets": 1000, 2019-11-25T10:25:57.199 INFO:teuthology.orchestra.run.smithi153.stdout: "subusers": [], 2019-11-25T10:25:57.199 INFO:teuthology.orchestra.run.smithi153.stdout: "keys": [ 2019-11-25T10:25:57.199 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T10:25:57.199 INFO:teuthology.orchestra.run.smithi153.stdout: "user": "foo.client.0", 2019-11-25T10:25:57.199 INFO:teuthology.orchestra.run.smithi153.stdout: "access_key": "LBFZNLAOSMYTHLEKZWHV", 2019-11-25T10:25:57.199 INFO:teuthology.orchestra.run.smithi153.stdout: "secret_key": "Ag9AAs4UYPrDJ3/6tjUEOOFIkemtkueOJpL0jWk8YVz4XEzIJ0sklw==" 2019-11-25T10:25:57.200 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T10:25:57.200 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T10:25:57.200 INFO:teuthology.orchestra.run.smithi153.stdout: "swift_keys": [], 2019-11-25T10:25:57.200 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": [], 2019-11-25T10:25:57.200 INFO:teuthology.orchestra.run.smithi153.stdout: "op_mask": "read, write, delete", 2019-11-25T10:25:57.200 INFO:teuthology.orchestra.run.smithi153.stdout: "default_placement": "", 2019-11-25T10:25:57.201 INFO:teuthology.orchestra.run.smithi153.stdout: "default_storage_class": "", 2019-11-25T10:25:57.201 INFO:teuthology.orchestra.run.smithi153.stdout: "placement_tags": [], 2019-11-25T10:25:57.201 INFO:teuthology.orchestra.run.smithi153.stdout: "bucket_quota": { 2019-11-25T10:25:57.201 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T10:25:57.201 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T10:25:57.201 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T10:25:57.202 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T10:25:57.202 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T10:25:57.202 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T10:25:57.202 INFO:teuthology.orchestra.run.smithi153.stdout: "user_quota": { 2019-11-25T10:25:57.202 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T10:25:57.202 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T10:25:57.203 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T10:25:57.203 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T10:25:57.203 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T10:25:57.203 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T10:25:57.203 INFO:teuthology.orchestra.run.smithi153.stdout: "temp_url_keys": [], 2019-11-25T10:25:57.203 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "rgw", 2019-11-25T10:25:57.203 INFO:teuthology.orchestra.run.smithi153.stdout: "mfa_ids": [] 2019-11-25T10:25:57.203 INFO:teuthology.orchestra.run.smithi153.stdout:} 2019-11-25T10:25:57.204 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:25:57.251 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.249+0000 7fe25955fec0 20 remove_watcher() i=0 2019-11-25T10:25:57.251 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.249+0000 7fe25955fec0 2 removed watcher, disabling cache 2019-11-25T10:25:57.315 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.313+0000 7fe25955fec0 20 remove_watcher() i=1 2019-11-25T10:25:57.372 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.370+0000 7fe25955fec0 20 remove_watcher() i=2 2019-11-25T10:25:57.424 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.421+0000 7fe25955fec0 20 remove_watcher() i=3 2019-11-25T10:25:57.466 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.463+0000 7fe25955fec0 20 remove_watcher() i=4 2019-11-25T10:25:57.503 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.500+0000 7fe25955fec0 20 remove_watcher() i=5 2019-11-25T10:25:57.565 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.563+0000 7fe25955fec0 20 remove_watcher() i=6 2019-11-25T10:25:57.607 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.605+0000 7fe25955fec0 20 remove_watcher() i=7 2019-11-25T10:25:57.618 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid foo.client.0 --totp-serial 9853903512 --totp-seed VW3E2ZWGJTQGUADVIY7TEELAUD3SV3VPYHR2YG37XVD444EFHEX46JNRNX6XKBT6 --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-11-25T10:25:57.758 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.755+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:57.788 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.785+0000 7f47a52a7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:57.788 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.785+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:57.799 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.794+0000 7f47a52a7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:57.799 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.794+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:57.806 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.804+0000 7f47a52a7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:57.806 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.804+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:57.830 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:57.828+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:58.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.058+0000 7f47a52a7ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T10:25:58.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.058+0000 7f47a52a7ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T10:25:58.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.058+0000 7f47a52a7ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T10:25:58.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.058+0000 7f47a52a7ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T10:25:58.061 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.058+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.075+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:58.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.075+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.091 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.088+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:58.091 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.089+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.107 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.104+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:58.107 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.104+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.119+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:58.122 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.119+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.134+0000 7f47a52a7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.134+0000 7f47a52a7ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T10:25:58.137 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.134+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.147 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.144+0000 7f47a52a7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:58.147 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.144+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.165 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.162+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:58.165 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.162+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.180 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.178+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:25:58.180 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.178+0000 7f47a52a7ec0 20 zonegroup default 2019-11-25T10:25:58.181 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.178+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.187+0000 7f47a52a7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.187+0000 7f47a52a7ec0 10 Cannot find current period zone using local zone 2019-11-25T10:25:58.190 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.187+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.195+0000 7f47a52a7ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:25:58.197 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.195+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.208+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:25:58.210 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.208+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:25:58.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.222+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:25:58.225 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.222+0000 7f47a52a7ec0 20 zone default found 2019-11-25T10:25:58.225 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.222+0000 7f47a52a7ec0 20 started sync module instance, tier type = 2019-11-25T10:25:58.225 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.222+0000 7f47a52a7ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T10:25:58.584 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.581+0000 7f47a52a7ec0 20 add_watcher() i=0 2019-11-25T10:25:58.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.581+0000 7f47a52a7ec0 20 add_watcher() i=1 2019-11-25T10:25:58.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.581+0000 7f47a52a7ec0 20 add_watcher() i=2 2019-11-25T10:25:58.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.581+0000 7f47a52a7ec0 20 add_watcher() i=3 2019-11-25T10:25:58.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.581+0000 7f47a52a7ec0 20 add_watcher() i=4 2019-11-25T10:25:58.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.581+0000 7f47a52a7ec0 20 add_watcher() i=5 2019-11-25T10:25:58.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.581+0000 7f47a52a7ec0 20 add_watcher() i=6 2019-11-25T10:25:58.611 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.609+0000 7f47a52a7ec0 20 add_watcher() i=7 2019-11-25T10:25:58.611 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.609+0000 7f47a52a7ec0 2 all 8 watchers are set, enabling cache 2019-11-25T10:25:58.612 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.609+0000 7f4762fdd700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:25:58.624 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.621+0000 7f47924d5700 0 -- >> v1:172.21.15.81:6789/0 conn(0x558c4c7e4850 legacy=0x558c4c7e6c70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T10:25:58.739 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:58.736+0000 7f4792cd6700 0 -- 172.21.15.153:0/2870369390 >> [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] conn(0x558c4c7f6470 msgr2=0x558c4c7f8890 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T10:25:59.982 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:59.979+0000 7f46d17fa700 20 reqs_thread_entry: start 2019-11-25T10:25:59.982 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:59.979+0000 7f46caffd700 20 reqs_thread_entry: start 2019-11-25T10:25:59.982 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:59.979+0000 7f47a52a7ec0 20 init_complete bucket index max shards: 0 2019-11-25T10:25:59.983 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:59.980+0000 7f46c97fa700 20 reqs_thread_entry: start 2019-11-25T10:25:59.983 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:59.980+0000 7f47a52a7ec0 20 get_system_obj_state: rctx=0x7fff1a9380b8 obj=default.rgw.meta:users.uid:foo.client.0 state=0x558c4c859fd0 s->prefetch_data=0 2019-11-25T10:25:59.983 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:25:59.980+0000 7f47a52a7ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-11-25T10:26:00.003 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.000+0000 7f47a52a7ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-11-25T10:26:00.003 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.000+0000 7f47a52a7ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T10:26:00.003 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.000+0000 7f47a52a7ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T10:26:00.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.000+0000 7f47a52a7ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T10:26:00.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.000+0000 7f47a52a7ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-25T10:26:00.004 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.000+0000 7f47a52a7ec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:00.019 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.016+0000 7f47a52a7ec0 20 rados_obj.operate() r=0 bl.length=432 2019-11-25T10:26:00.019 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.016+0000 7f47a52a7ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-11-25T10:26:00.019 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.016+0000 7f47a52a7ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T10:26:00.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.212+0000 7f47a52a7ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-25T10:26:00.215 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.212+0000 7f47a52a7ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T10:26:00.215 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.212+0000 7f47a52a7ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=631 2019-11-25T10:26:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.227+0000 7f47737fe700 10 RGWWatcher::handle_notify() notify_id 592705486909 cookie 94061067020640 notifier 4649 bl.length()=631 2019-11-25T10:26:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.228+0000 7f47737fe700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-25T10:26:00.230 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.228+0000 7f47737fe700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T10:26:00.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.722+0000 7f47a52a7ec0 20 remove_watcher() i=0 2019-11-25T10:26:00.725 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.722+0000 7f47a52a7ec0 2 removed watcher, disabling cache 2019-11-25T10:26:00.779 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.777+0000 7f47a52a7ec0 20 remove_watcher() i=1 2019-11-25T10:26:00.834 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.831+0000 7f47a52a7ec0 20 remove_watcher() i=2 2019-11-25T10:26:00.895 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.893+0000 7f47a52a7ec0 20 remove_watcher() i=3 2019-11-25T10:26:00.942 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.939+0000 7f47a52a7ec0 20 remove_watcher() i=4 2019-11-25T10:26:00.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:00.976+0000 7f47a52a7ec0 20 remove_watcher() i=5 2019-11-25T10:26:01.020 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.017+0000 7f47a52a7ec0 20 remove_watcher() i=6 2019-11-25T10:26:01.058 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.055+0000 7f47a52a7ec0 20 remove_watcher() i=7 2019-11-25T10:26:01.064 DEBUG:tasks.s3tests:Creating user testx$tenanteduser.client.0 on client.0 2019-11-25T10:26:01.065 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user create --uid 'testx$tenanteduser.client.0' --display-name 'Mr. testx$tenanteduser.client.0' --access-key ZWMLDUAQBGMKDPFMGHKS --secret kck8XWRzCp2rtGYrhHrGuibzmcG767a86ONNWDlvWCCeyhYZgWruMQ== --email 'testx$tenanteduser.client.0+test@test.test' --cluster ceph 2019-11-25T10:26:01.187 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.184+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.207 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.205+0000 7f7db762bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:01.207 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.205+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.212+0000 7f7db762bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:01.214 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.212+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.219+0000 7f7db762bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:01.221 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.219+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.236+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:01.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.400+0000 7f7db762bec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T10:26:01.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.400+0000 7f7db762bec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T10:26:01.403 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.400+0000 7f7db762bec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T10:26:01.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.400+0000 7f7db762bec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T10:26:01.404 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.400+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.415 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.412+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:01.415 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.412+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.431+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:26:01.434 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.431+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.447 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.445+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:01.447 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.445+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.463 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.460+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:26:01.463 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.460+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.478 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.475+0000 7f7db762bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:01.478 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.475+0000 7f7db762bec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T10:26:01.479 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.475+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.486+0000 7f7db762bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:01.488 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.486+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.509 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.506+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:01.509 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.506+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.518+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:26:01.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.518+0000 7f7db762bec0 20 zonegroup default 2019-11-25T10:26:01.521 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.518+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.529 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.527+0000 7f7db762bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:01.530 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.527+0000 7f7db762bec0 10 Cannot find current period zone using local zone 2019-11-25T10:26:01.530 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.527+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.534+0000 7f7db762bec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:01.537 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.535+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.550 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.547+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:01.550 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.547+0000 7f7db762bec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:01.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.563+0000 7f7db762bec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:26:01.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.563+0000 7f7db762bec0 20 zone default found 2019-11-25T10:26:01.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.563+0000 7f7db762bec0 20 started sync module instance, tier type = 2019-11-25T10:26:01.566 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.563+0000 7f7db762bec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T10:26:01.977 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.975+0000 7f7db762bec0 20 add_watcher() i=0 2019-11-25T10:26:01.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.975+0000 7f7db762bec0 20 add_watcher() i=1 2019-11-25T10:26:01.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.975+0000 7f7db762bec0 20 add_watcher() i=2 2019-11-25T10:26:01.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.975+0000 7f7db762bec0 20 add_watcher() i=3 2019-11-25T10:26:01.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.975+0000 7f7db762bec0 20 add_watcher() i=4 2019-11-25T10:26:01.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.975+0000 7f7db762bec0 20 add_watcher() i=5 2019-11-25T10:26:01.978 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:01.975+0000 7f7db762bec0 20 add_watcher() i=6 2019-11-25T10:26:02.007 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:02.004+0000 7f7db762bec0 20 add_watcher() i=7 2019-11-25T10:26:02.007 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:02.004+0000 7f7db762bec0 2 all 8 watchers are set, enabling cache 2019-11-25T10:26:02.008 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:02.004+0000 7f7d757e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:26:02.009 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:02.007+0000 7f7d9c058700 0 -- 172.21.15.153:0/2251283691 >> v2:172.21.15.153:3300/0 conn(0x55c1ae552560 msgr2=0x55c1ae554a70 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T10:26:02.048 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:02.045+0000 7f7da4859700 0 -- 172.21.15.153:0/288676357 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x55c1ae555b50 msgr2=0x55c1ae557f70 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T10:26:03.543 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.541+0000 7f7ce77fe700 20 reqs_thread_entry: start 2019-11-25T10:26:03.544 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.541+0000 7f7ce57fa700 20 reqs_thread_entry: start 2019-11-25T10:26:03.544 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.541+0000 7f7db762bec0 20 init_complete bucket index max shards: 0 2019-11-25T10:26:03.544 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.541+0000 7f7cd37fe700 20 reqs_thread_entry: start 2019-11-25T10:26:03.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.541+0000 7f7db762bec0 20 get_system_obj_state: rctx=0x7ffda41be838 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55c1ae5ca360 s->prefetch_data=0 2019-11-25T10:26:03.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.541+0000 7f7db762bec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-11-25T10:26:03.553 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.551+0000 7f7db762bec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x0 2019-11-25T10:26:03.554 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.551+0000 7f7db762bec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T10:26:03.554 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.551+0000 7f7db762bec0 20 get_system_obj_state: rctx=0x7ffda41be848 obj=default.rgw.meta:users.email:testx$tenanteduser.client.0+test@test.test state=0x55c1ae5cc110 s->prefetch_data=0 2019-11-25T10:26:03.554 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.551+0000 7f7db762bec0 10 cache get: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test : miss 2019-11-25T10:26:03.562 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.560+0000 7f7db762bec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x0 2019-11-25T10:26:03.562 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.560+0000 7f7db762bec0 10 adding default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-11-25T10:26:03.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.560+0000 7f7db762bec0 20 get_system_obj_state: rctx=0x7ffda41be848 obj=default.rgw.meta:users.keys:ZWMLDUAQBGMKDPFMGHKS state=0x55c1ae5cc110 s->prefetch_data=0 2019-11-25T10:26:03.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.560+0000 7f7db762bec0 10 cache get: name=default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS : miss 2019-11-25T10:26:03.571 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.568+0000 7f7db762bec0 10 cache put: name=default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS info.flags=0x0 2019-11-25T10:26:03.571 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.568+0000 7f7db762bec0 10 adding default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS to cache LRU end 2019-11-25T10:26:03.571 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.568+0000 7f7db762bec0 20 get_system_obj_state: rctx=0x7ffda41be458 obj=default.rgw.meta:users.keys:ZWMLDUAQBGMKDPFMGHKS state=0x55c1ae5cc110 s->prefetch_data=0 2019-11-25T10:26:03.571 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.568+0000 7f7db762bec0 10 cache get: name=default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS : type miss (requested=0x6, cached=0x0) 2019-11-25T10:26:03.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.576+0000 7f7db762bec0 10 cache put: name=default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS info.flags=0x0 2019-11-25T10:26:03.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.576+0000 7f7db762bec0 10 moving default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS to cache LRU end 2019-11-25T10:26:03.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.576+0000 7f7db762bec0 20 get_system_obj_state: rctx=0x7ffda41be5d8 obj=default.rgw.meta:users.keys:ZWMLDUAQBGMKDPFMGHKS state=0x55c1ae5cc110 s->prefetch_data=0 2019-11-25T10:26:03.579 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.576+0000 7f7db762bec0 10 cache get: name=default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS : type miss (requested=0x6, cached=0x0) 2019-11-25T10:26:03.586 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.584+0000 7f7db762bec0 10 cache put: name=default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS info.flags=0x0 2019-11-25T10:26:03.586 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.584+0000 7f7db762bec0 10 moving default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS to cache LRU end 2019-11-25T10:26:03.635 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.633+0000 7f7db762bec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-11-25T10:26:03.635 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.633+0000 7f7db762bec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T10:26:03.635 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.633+0000 7f7db762bec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=691 2019-11-25T10:26:03.648 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.645+0000 7f7d9a054700 10 RGWWatcher::handle_notify() notify_id 592705486916 cookie 94290341780416 notifier 4661 bl.length()=691 2019-11-25T10:26:03.648 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.645+0000 7f7d9a054700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-11-25T10:26:03.648 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.645+0000 7f7d9a054700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T10:26:03.917 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.915+0000 7f7db762bec0 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-11-25T10:26:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.915+0000 7f7db762bec0 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-11-25T10:26:03.918 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.915+0000 7f7db762bec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=224 2019-11-25T10:26:03.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.938+0000 7f7d9a054700 10 RGWWatcher::handle_notify() notify_id 592705486910 cookie 94290341787552 notifier 4661 bl.length()=224 2019-11-25T10:26:03.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.938+0000 7f7d9a054700 10 cache put: name=default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test info.flags=0x7 2019-11-25T10:26:03.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:03.938+0000 7f7d9a054700 10 moving default.rgw.meta+users.email+testx$tenanteduser.client.0+test@test.test to cache LRU end 2019-11-25T10:26:04.292 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.289+0000 7f7db762bec0 10 cache put: name=default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS info.flags=0x7 2019-11-25T10:26:04.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.289+0000 7f7db762bec0 10 moving default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS to cache LRU end 2019-11-25T10:26:04.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.289+0000 7f7db762bec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=201 2019-11-25T10:26:04.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.308+0000 7f7d9a054700 10 RGWWatcher::handle_notify() notify_id 592705486917 cookie 94290341780416 notifier 4661 bl.length()=201 2019-11-25T10:26:04.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.308+0000 7f7d9a054700 10 cache put: name=default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS info.flags=0x7 2019-11-25T10:26:04.311 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.308+0000 7f7d9a054700 10 moving default.rgw.meta+users.keys+ZWMLDUAQBGMKDPFMGHKS to cache LRU end 2019-11-25T10:26:04.740 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2019-11-25T10:26:04.741 INFO:teuthology.orchestra.run.smithi153.stdout: "user_id": "testx$tenanteduser.client.0", 2019-11-25T10:26:04.741 INFO:teuthology.orchestra.run.smithi153.stdout: "display_name": "Mr. testx$tenanteduser.client.0", 2019-11-25T10:26:04.741 INFO:teuthology.orchestra.run.smithi153.stdout: "email": "testx$tenanteduser.client.0+test@test.test", 2019-11-25T10:26:04.741 INFO:teuthology.orchestra.run.smithi153.stdout: "suspended": 0, 2019-11-25T10:26:04.741 INFO:teuthology.orchestra.run.smithi153.stdout: "max_buckets": 1000, 2019-11-25T10:26:04.741 INFO:teuthology.orchestra.run.smithi153.stdout: "subusers": [], 2019-11-25T10:26:04.742 INFO:teuthology.orchestra.run.smithi153.stdout: "keys": [ 2019-11-25T10:26:04.742 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T10:26:04.742 INFO:teuthology.orchestra.run.smithi153.stdout: "user": "testx$tenanteduser.client.0", 2019-11-25T10:26:04.742 INFO:teuthology.orchestra.run.smithi153.stdout: "access_key": "ZWMLDUAQBGMKDPFMGHKS", 2019-11-25T10:26:04.742 INFO:teuthology.orchestra.run.smithi153.stdout: "secret_key": "kck8XWRzCp2rtGYrhHrGuibzmcG767a86ONNWDlvWCCeyhYZgWruMQ==" 2019-11-25T10:26:04.742 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T10:26:04.743 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T10:26:04.743 INFO:teuthology.orchestra.run.smithi153.stdout: "swift_keys": [], 2019-11-25T10:26:04.743 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": [], 2019-11-25T10:26:04.743 INFO:teuthology.orchestra.run.smithi153.stdout: "op_mask": "read, write, delete", 2019-11-25T10:26:04.743 INFO:teuthology.orchestra.run.smithi153.stdout: "default_placement": "", 2019-11-25T10:26:04.743 INFO:teuthology.orchestra.run.smithi153.stdout: "default_storage_class": "", 2019-11-25T10:26:04.743 INFO:teuthology.orchestra.run.smithi153.stdout: "placement_tags": [], 2019-11-25T10:26:04.744 INFO:teuthology.orchestra.run.smithi153.stdout: "bucket_quota": { 2019-11-25T10:26:04.744 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T10:26:04.744 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T10:26:04.744 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T10:26:04.744 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T10:26:04.744 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T10:26:04.745 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T10:26:04.745 INFO:teuthology.orchestra.run.smithi153.stdout: "user_quota": { 2019-11-25T10:26:04.745 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T10:26:04.745 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T10:26:04.745 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T10:26:04.745 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T10:26:04.746 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T10:26:04.746 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T10:26:04.746 INFO:teuthology.orchestra.run.smithi153.stdout: "temp_url_keys": [], 2019-11-25T10:26:04.746 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "rgw", 2019-11-25T10:26:04.746 INFO:teuthology.orchestra.run.smithi153.stdout: "mfa_ids": [] 2019-11-25T10:26:04.746 INFO:teuthology.orchestra.run.smithi153.stdout:} 2019-11-25T10:26:04.746 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:04.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.791+0000 7f7db762bec0 20 remove_watcher() i=0 2019-11-25T10:26:04.794 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.791+0000 7f7db762bec0 2 removed watcher, disabling cache 2019-11-25T10:26:04.865 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.863+0000 7f7db762bec0 20 remove_watcher() i=1 2019-11-25T10:26:04.916 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.913+0000 7f7db762bec0 20 remove_watcher() i=2 2019-11-25T10:26:04.967 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:04.965+0000 7f7db762bec0 20 remove_watcher() i=3 2019-11-25T10:26:05.018 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.015+0000 7f7db762bec0 20 remove_watcher() i=4 2019-11-25T10:26:05.076 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.073+0000 7f7db762bec0 20 remove_watcher() i=5 2019-11-25T10:26:05.130 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.127+0000 7f7db762bec0 20 remove_watcher() i=6 2019-11-25T10:26:05.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.173+0000 7f7db762bec0 20 remove_watcher() i=7 2019-11-25T10:26:05.184 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 mfa create --uid 'testx$tenanteduser.client.0' --totp-serial 7765051065 --totp-seed TAX4KZFSOKNFJ2ZEZ6K2R7JQQCHU3A4OAU7ULDT37YIK7GCWHFGWBWUSBHMK2FUH --totp-seconds 5 --totp-window 8 --totp-seed-type base32 --cluster ceph 2019-11-25T10:26:05.323 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.320+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.357+0000 7fb96104cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:05.360 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.358+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.367 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.365+0000 7fb96104cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:05.367 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.365+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.374+0000 7fb96104cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:05.377 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.374+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.393 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.390+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:05.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.544+0000 7fb96104cec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T10:26:05.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.544+0000 7fb96104cec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T10:26:05.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.544+0000 7fb96104cec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T10:26:05.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.544+0000 7fb96104cec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T10:26:05.547 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.544+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.554 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.552+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:05.554 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.552+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.562 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.560+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:26:05.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.560+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.573 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.571+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:05.574 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.571+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.584 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.581+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:26:05.584 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.581+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.590 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.587+0000 7fb96104cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:05.590 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.587+0000 7fb96104cec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T10:26:05.590 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.587+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.598 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.595+0000 7fb96104cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:05.598 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.595+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.606 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.604+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:05.607 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.604+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.613+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T10:26:05.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.613+0000 7fb96104cec0 20 zonegroup default 2019-11-25T10:26:05.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.613+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.620 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.618+0000 7fb96104cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.618+0000 7fb96104cec0 10 Cannot find current period zone using local zone 2019-11-25T10:26:05.621 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.618+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.626 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.623+0000 7fb96104cec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T10:26:05.626 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.623+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.635 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.632+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T10:26:05.635 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.632+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:05.646 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.643+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T10:26:05.646 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.643+0000 7fb96104cec0 20 zone default found 2019-11-25T10:26:05.646 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.643+0000 7fb96104cec0 20 started sync module instance, tier type = 2019-11-25T10:26:05.646 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:05.643+0000 7fb96104cec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T10:26:06.016 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.013+0000 7fb96104cec0 20 add_watcher() i=0 2019-11-25T10:26:06.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.013+0000 7fb96104cec0 20 add_watcher() i=1 2019-11-25T10:26:06.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.013+0000 7fb96104cec0 20 add_watcher() i=2 2019-11-25T10:26:06.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.013+0000 7fb96104cec0 20 add_watcher() i=3 2019-11-25T10:26:06.224 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.013+0000 7fb96104cec0 20 add_watcher() i=4 2019-11-25T10:26:06.225 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.013+0000 7fb96104cec0 20 add_watcher() i=5 2019-11-25T10:26:06.225 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.013+0000 7fb96104cec0 20 add_watcher() i=6 2019-11-25T10:26:06.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.040+0000 7fb96104cec0 20 add_watcher() i=7 2019-11-25T10:26:06.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.040+0000 7fb96104cec0 2 all 8 watchers are set, enabling cache 2019-11-25T10:26:06.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.040+0000 7fb91dfeb700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T10:26:06.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.046+0000 7fb94ea7b700 0 -- 172.21.15.153:0/3849830006 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5578ccc84340 msgr2=0x5578ccc875a0 secure :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T10:26:06.252 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.249+0000 7fb94ea7b700 0 --2- 172.21.15.153:0/3849830006 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5578ccc84340 0x5578ccc875a0 unknown :-1 s=AUTH_CONNECTING pgs=0 cs=0 l=0 rx=0 tx=0).send_auth_request get_initial_auth_request returned -2 2019-11-25T10:26:06.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:06.296+0000 7fb94ea7b700 0 -- 172.21.15.153:0/2270567570 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x7fb8900028f0 msgr2=0x7fb890002e90 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T10:26:07.835 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.831+0000 7fb88ffff700 20 reqs_thread_entry: start 2019-11-25T10:26:07.835 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.831+0000 7fb88dffb700 20 reqs_thread_entry: start 2019-11-25T10:26:07.835 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.831+0000 7fb96104cec0 20 init_complete bucket index max shards: 0 2019-11-25T10:26:07.836 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.832+0000 7fb87bfff700 20 reqs_thread_entry: start 2019-11-25T10:26:07.836 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.832+0000 7fb96104cec0 20 get_system_obj_state: rctx=0x7ffd867d9f58 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x5578cccf8c50 s->prefetch_data=0 2019-11-25T10:26:07.836 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.832+0000 7fb96104cec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-11-25T10:26:07.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.854+0000 7fb96104cec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-11-25T10:26:07.857 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.854+0000 7fb96104cec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T10:26:07.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.854+0000 7fb96104cec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T10:26:07.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.854+0000 7fb96104cec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T10:26:07.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.854+0000 7fb96104cec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-25T10:26:07.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.854+0000 7fb96104cec0 20 rados->read ofs=0 len=0 2019-11-25T10:26:07.879 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.876+0000 7fb96104cec0 20 rados_obj.operate() r=0 bl.length=491 2019-11-25T10:26:07.880 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.876+0000 7fb96104cec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-11-25T10:26:07.880 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:07.876+0000 7fb96104cec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T10:26:08.062 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.059+0000 7fb96104cec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-11-25T10:26:08.063 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.059+0000 7fb96104cec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T10:26:08.063 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.059+0000 7fb96104cec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=705 2019-11-25T10:26:08.082 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.079+0000 7fb93f7fe700 10 RGWWatcher::handle_notify() notify_id 592705486918 cookie 93977320057968 notifier 4905 bl.length()=705 2019-11-25T10:26:08.082 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.079+0000 7fb93f7fe700 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x17 2019-11-25T10:26:08.082 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.079+0000 7fb93f7fe700 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T10:26:08.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.596+0000 7fb96104cec0 20 remove_watcher() i=0 2019-11-25T10:26:08.600 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.596+0000 7fb96104cec0 2 removed watcher, disabling cache 2019-11-25T10:26:08.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.662+0000 7fb96104cec0 20 remove_watcher() i=1 2019-11-25T10:26:08.741 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.737+0000 7fb96104cec0 20 remove_watcher() i=2 2019-11-25T10:26:08.798 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.794+0000 7fb96104cec0 20 remove_watcher() i=3 2019-11-25T10:26:08.846 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.843+0000 7fb96104cec0 20 remove_watcher() i=4 2019-11-25T10:26:08.899 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.896+0000 7fb96104cec0 20 remove_watcher() i=5 2019-11-25T10:26:08.956 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:08.953+0000 7fb96104cec0 20 remove_watcher() i=6 2019-11-25T10:26:09.014 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T10:26:09.010+0000 7fb96104cec0 20 remove_watcher() i=7 2019-11-25T10:26:09.021 INFO:tasks.s3tests:Configuring s3-tests... 2019-11-25T10:26:09.021 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest/s3-tests && ./bootstrap 2019-11-25T10:26:14.340 INFO:teuthology.orchestra.run.smithi153.stderr:./bootstrap: missing required RPM packages. Installing via sudo. 2019-11-25T10:26:14.461 INFO:teuthology.orchestra.run.smithi153.stdout:Loaded plugins: fastestmirror, langpacks, priorities 2019-11-25T10:26:14.503 INFO:teuthology.orchestra.run.smithi153.stdout:Loading mirror speeds from cached hostfile 2019-11-25T10:26:14.504 INFO:teuthology.orchestra.run.smithi153.stdout: * base: mirror.cc.columbia.edu 2019-11-25T10:26:14.505 INFO:teuthology.orchestra.run.smithi153.stdout: * epel: ftp.linux.ncsu.edu 2019-11-25T10:26:14.505 INFO:teuthology.orchestra.run.smithi153.stdout: * extras: mirror.jaleco.com 2019-11-25T10:26:14.506 INFO:teuthology.orchestra.run.smithi153.stdout: * updates: mirror.atlanticmetro.net 2019-11-25T10:26:15.162 INFO:teuthology.orchestra.run.smithi153.stdout:7 packages excluded due to repository priority protections 2019-11-25T10:26:15.510 INFO:teuthology.orchestra.run.smithi153.stdout:Resolving Dependencies 2019-11-25T10:26:15.511 INFO:teuthology.orchestra.run.smithi153.stdout:--> Running transaction check 2019-11-25T10:26:15.511 INFO:teuthology.orchestra.run.smithi153.stdout:---> Package libffi-devel.x86_64 0:3.0.13-18.el7 will be installed 2019-11-25T10:26:15.902 INFO:teuthology.orchestra.run.smithi153.stdout:--> Finished Dependency Resolution 2019-11-25T10:26:16.139 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:16.139 INFO:teuthology.orchestra.run.smithi153.stdout:Dependencies Resolved 2019-11-25T10:26:16.140 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:16.140 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T10:26:16.140 INFO:teuthology.orchestra.run.smithi153.stdout: Package Arch Version Repository Size 2019-11-25T10:26:16.140 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T10:26:16.141 INFO:teuthology.orchestra.run.smithi153.stdout:Installing: 2019-11-25T10:26:16.141 INFO:teuthology.orchestra.run.smithi153.stdout: libffi-devel x86_64 3.0.13-18.el7 base 23 k 2019-11-25T10:26:16.141 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:16.141 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction Summary 2019-11-25T10:26:16.142 INFO:teuthology.orchestra.run.smithi153.stdout:================================================================================ 2019-11-25T10:26:16.142 INFO:teuthology.orchestra.run.smithi153.stdout:Install 1 Package 2019-11-25T10:26:16.143 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:16.143 INFO:teuthology.orchestra.run.smithi153.stdout:Total download size: 23 k 2019-11-25T10:26:16.143 INFO:teuthology.orchestra.run.smithi153.stdout:Installed size: 27 k 2019-11-25T10:26:16.143 INFO:teuthology.orchestra.run.smithi153.stdout:Downloading packages: 2019-11-25T10:26:16.256 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction check 2019-11-25T10:26:16.259 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction test 2019-11-25T10:26:16.261 INFO:teuthology.orchestra.run.smithi153.stdout:Transaction test succeeded 2019-11-25T10:26:16.261 INFO:teuthology.orchestra.run.smithi153.stdout:Running transaction 2019-11-25T10:26:16.811 INFO:teuthology.orchestra.run.smithi153.stdout: Installing : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-11-25T10:26:17.264 INFO:teuthology.orchestra.run.smithi153.stdout: Verifying : libffi-devel-3.0.13-18.el7.x86_64 1/1 2019-11-25T10:26:17.265 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:17.265 INFO:teuthology.orchestra.run.smithi153.stdout:Installed: 2019-11-25T10:26:17.265 INFO:teuthology.orchestra.run.smithi153.stdout: libffi-devel.x86_64 0:3.0.13-18.el7 2019-11-25T10:26:17.265 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:17.265 INFO:teuthology.orchestra.run.smithi153.stdout:Complete! 2019-11-25T10:26:17.457 INFO:teuthology.orchestra.run.smithi153.stdout:New python executable in /home/ubuntu/cephtest/s3-tests/virtualenv/bin/python 2019-11-25T10:26:18.694 INFO:teuthology.orchestra.run.smithi153.stdout:Installing setuptools, pip, wheel...done. 2019-11-25T10:26:19.022 INFO:teuthology.orchestra.run.smithi153.stderr:Cache entry deserialization failed, entry ignored 2019-11-25T10:26:19.167 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting pip 2019-11-25T10:26:19.245 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-11-25T10:26:19.311 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: pip 2019-11-25T10:26:19.311 INFO:teuthology.orchestra.run.smithi153.stdout: Found existing installation: pip 9.0.1 2019-11-25T10:26:19.369 INFO:teuthology.orchestra.run.smithi153.stdout: Uninstalling pip-9.0.1: 2019-11-25T10:26:19.415 INFO:teuthology.orchestra.run.smithi153.stdout: Successfully uninstalled pip-9.0.1 2019-11-25T10:26:20.176 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed pip-19.3.1 2019-11-25T10:26:20.518 INFO:teuthology.orchestra.run.smithi153.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-25T10:26:21.034 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting setuptools==32.3.1 2019-11-25T10:26:21.038 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-11-25T10:26:21.101 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: setuptools 2019-11-25T10:26:21.102 INFO:teuthology.orchestra.run.smithi153.stdout: Found existing installation: setuptools 28.8.0 2019-11-25T10:26:21.128 INFO:teuthology.orchestra.run.smithi153.stdout: Uninstalling setuptools-28.8.0: 2019-11-25T10:26:21.143 INFO:teuthology.orchestra.run.smithi153.stdout: Successfully uninstalled setuptools-28.8.0 2019-11-25T10:26:21.309 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed setuptools-32.3.1 2019-11-25T10:26:21.868 INFO:teuthology.orchestra.run.smithi153.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-25T10:26:22.057 INFO:teuthology.orchestra.run.smithi153.stdout:Processing /home/ubuntu/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030/PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl 2019-11-25T10:26:22.123 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting nose>=1.0.0 2019-11-25T10:26:22.126 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-11-25T10:26:22.282 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting boto>=2.6.0 2019-11-25T10:26:22.290 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-11-25T10:26:22.767 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting boto3>=1.0.0 2019-11-25T10:26:22.806 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/9b/11/7e6470f5d7d7d23fc5eaae64f8a3f4b844ca08234cc3207df027267b65c4/boto3-1.10.26-py2.py3-none-any.whl (128kB) 2019-11-25T10:26:22.928 INFO:teuthology.orchestra.run.smithi153.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-11-25T10:26:23.185 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting gevent>=1.0 2019-11-25T10:26:23.200 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-25T10:26:23.458 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting isodate>=0.4.4 2019-11-25T10:26:23.468 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl (45kB) 2019-11-25T10:26:23.631 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting requests>=0.14.0 2019-11-25T10:26:23.641 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/51/bd/23c926cd341ea6b7dd0b2a00aba99ae0f828be89d72b2190f27c11d4b7fb/requests-2.22.0-py2.py3-none-any.whl (57kB) 2019-11-25T10:26:23.907 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting pytz>=2011k 2019-11-25T10:26:23.917 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/e7/f9/f0b53f88060247251bf481fa6ea62cd0d25bf1b11a87888e53ce5b7c8ad2/pytz-2019.3-py2.py3-none-any.whl (509kB) 2019-11-25T10:26:24.168 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting ordereddict 2019-11-25T10:26:24.178 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/53/25/ef88e8e45db141faa9598fbf7ad0062df8f50f881a36ed6a0073e1572126/ordereddict-1.1.tar.gz 2019-11-25T10:26:24.497 INFO:teuthology.orchestra.run.smithi153.stdout:Processing /home/ubuntu/.cache/pip/wheels/51/ab/65/f318b4271019823cf7d7c178ac76a8ea4d6e6271fe9f5ba488/httplib2-0.14.0-cp27-none-any.whl 2019-11-25T10:26:24.881 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting lxml 2019-11-25T10:26:24.896 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/e4/f4/65d145cd6917131826050b0479be35aaccba2847b7f80fc4afc6bec6616b/lxml-4.4.1-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-25T10:26:25.104 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting jmespath<1.0.0,>=0.7.1 2019-11-25T10:26:25.115 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/83/94/7179c3832a6d45b266ddb2aac329e101367fbdb11f425f13771d27f225bb/jmespath-0.9.4-py2.py3-none-any.whl 2019-11-25T10:26:25.862 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting botocore<1.14.0,>=1.13.26 2019-11-25T10:26:25.873 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/8a/93/ea2ec042794dfda186348df02c6057223a8bbc21c055124fbe3e16925441/botocore-1.13.26-py2.py3-none-any.whl (5.6MB) 2019-11-25T10:26:26.431 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting s3transfer<0.3.0,>=0.2.0 2019-11-25T10:26:26.444 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/16/8a/1fc3dba0c4923c2a76e1ff0d52b305c44606da63f718d14d3231e21c51b0/s3transfer-0.2.1-py2.py3-none-any.whl (70kB) 2019-11-25T10:26:26.637 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-11-25T10:26:26.639 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-25T10:26:26.708 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting six 2019-11-25T10:26:26.719 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-11-25T10:26:26.801 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting chardet<3.1.0,>=3.0.2 2019-11-25T10:26:26.815 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/bc/a9/01ffebfb562e4274b6487b4bb1ddec7ca55ec7510b22e4c51f14098443b8/chardet-3.0.4-py2.py3-none-any.whl (133kB) 2019-11-25T10:26:26.912 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting idna<2.9,>=2.5 2019-11-25T10:26:26.922 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/14/2c/cd551d81dbe15200be1cf41cd03869a46fe7226e7450af7a6545bfc474c9/idna-2.8-py2.py3-none-any.whl (58kB) 2019-11-25T10:26:27.068 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 2019-11-25T10:26:27.079 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/b4/40/a9837291310ee1ccc242ceb6ebfd9eb21539649f193a7c8c86ba15b98539/urllib3-1.25.7-py2.py3-none-any.whl (125kB) 2019-11-25T10:26:27.233 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting certifi>=2017.4.17 2019-11-25T10:26:27.244 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/18/b0/8146a4f8dd402f60744fa380bc73ca47303cccf8b9190fd16a827281eac2/certifi-2019.9.11-py2.py3-none-any.whl (154kB) 2019-11-25T10:26:27.346 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting docutils<0.16,>=0.10 2019-11-25T10:26:27.358 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/3a/dc/bf2b15d1fa15a6f7a9e77a61b74ecbbae7258558fcda8ffc9a6638a6b327/docutils-0.15.2-py2-none-any.whl (548kB) 2019-11-25T10:26:27.539 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting python-dateutil<2.8.1,>=2.1; python_version >= "2.7" 2019-11-25T10:26:27.551 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/41/17/c62faccbfbd163c7f57f3844689e3a78bae1f403648a6afb1d0866d87fbb/python_dateutil-2.8.0-py2.py3-none-any.whl (226kB) 2019-11-25T10:26:27.673 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting futures<4.0.0,>=2.2.0; python_version == "2.6" or python_version == "2.7" 2019-11-25T10:26:27.684 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/d8/a6/f46ae3f1da0cd4361c344888f59ec2f5785e69c872e175a748ef6071cdb5/futures-3.3.0-py2-none-any.whl 2019-11-25T10:26:27.710 INFO:teuthology.orchestra.run.smithi153.stdout:Building wheels for collected packages: ordereddict 2019-11-25T10:26:27.711 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for ordereddict (setup.py): started 2019-11-25T10:26:27.927 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for ordereddict (setup.py): finished with status 'done' 2019-11-25T10:26:27.928 INFO:teuthology.orchestra.run.smithi153.stdout: Created wheel for ordereddict: filename=ordereddict-1.1-cp27-none-any.whl size=3327 sha256=0777a98e1b4c8dd450090c9310a7fdddc0273b39875486dacf02da655ee6fb89 2019-11-25T10:26:27.928 INFO:teuthology.orchestra.run.smithi153.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/78/d3/a8/9f52c7a389a0ffba6f575a3886b5f7a55461fb6ec34aa5fd38 2019-11-25T10:26:27.932 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully built ordereddict 2019-11-25T10:26:28.034 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: PyYAML, nose, boto, jmespath, docutils, six, python-dateutil, urllib3, botocore, futures, s3transfer, boto3, bunch, greenlet, gevent, isodate, chardet, idna, certifi, requests, pytz, ordereddict, httplib2, lxml 2019-11-25T10:26:30.316 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed PyYAML-5.1.2 boto-2.49.0 boto3-1.10.26 botocore-1.13.26 bunch-1.0.1 certifi-2019.9.11 chardet-3.0.4 docutils-0.15.2 futures-3.3.0 gevent-1.4.0 greenlet-0.4.15 httplib2-0.14.0 idna-2.8 isodate-0.6.0 jmespath-0.9.4 lxml-4.4.1 nose-1.3.7 ordereddict-1.1 python-dateutil-2.8.0 pytz-2019.3 requests-2.22.0 s3transfer-0.2.1 six-1.13.0 urllib3-1.25.7 2019-11-25T10:26:30.634 INFO:teuthology.orchestra.run.smithi153.stdout:running develop 2019-11-25T10:26:30.684 INFO:teuthology.orchestra.run.smithi153.stdout:running egg_info 2019-11-25T10:26:30.685 INFO:teuthology.orchestra.run.smithi153.stdout:creating s3tests.egg-info 2019-11-25T10:26:30.685 INFO:teuthology.orchestra.run.smithi153.stdout:writing requirements to s3tests.egg-info/requires.txt 2019-11-25T10:26:30.685 INFO:teuthology.orchestra.run.smithi153.stdout:writing s3tests.egg-info/PKG-INFO 2019-11-25T10:26:30.685 INFO:teuthology.orchestra.run.smithi153.stdout:writing top-level names to s3tests.egg-info/top_level.txt 2019-11-25T10:26:30.685 INFO:teuthology.orchestra.run.smithi153.stdout:writing dependency_links to s3tests.egg-info/dependency_links.txt 2019-11-25T10:26:30.686 INFO:teuthology.orchestra.run.smithi153.stdout:writing entry points to s3tests.egg-info/entry_points.txt 2019-11-25T10:26:30.686 INFO:teuthology.orchestra.run.smithi153.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-11-25T10:26:30.688 INFO:teuthology.orchestra.run.smithi153.stdout:reading manifest file 's3tests.egg-info/SOURCES.txt' 2019-11-25T10:26:30.689 INFO:teuthology.orchestra.run.smithi153.stdout:writing manifest file 's3tests.egg-info/SOURCES.txt' 2019-11-25T10:26:30.690 INFO:teuthology.orchestra.run.smithi153.stdout:running build_ext 2019-11-25T10:26:30.690 INFO:teuthology.orchestra.run.smithi153.stdout:Creating /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages/s3tests.egg-link (link to .) 2019-11-25T10:26:30.690 INFO:teuthology.orchestra.run.smithi153.stdout:Adding s3tests 0.0.1 to easy-install.pth file 2019-11-25T10:26:30.693 INFO:teuthology.orchestra.run.smithi153.stdout:Installing s3tests-fuzz-headers script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-25T10:26:30.693 INFO:teuthology.orchestra.run.smithi153.stdout:Installing s3tests-generate-objects script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-25T10:26:30.693 INFO:teuthology.orchestra.run.smithi153.stdout:Installing s3tests-test-roundtrip script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-25T10:26:30.693 INFO:teuthology.orchestra.run.smithi153.stdout:Installing s3tests-analysis-rwstats script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-25T10:26:30.694 INFO:teuthology.orchestra.run.smithi153.stdout:Installing s3tests-test-readwrite script to /home/ubuntu/cephtest/s3-tests/virtualenv/bin 2019-11-25T10:26:30.694 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.694 INFO:teuthology.orchestra.run.smithi153.stdout:Installed /home/ubuntu/cephtest/s3-tests 2019-11-25T10:26:30.694 INFO:teuthology.orchestra.run.smithi153.stdout:Processing dependencies for s3tests==0.0.1 2019-11-25T10:26:30.744 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for isodate==0.6.0 2019-11-25T10:26:30.745 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: isodate 0.6.0 2019-11-25T10:26:30.745 INFO:teuthology.orchestra.run.smithi153.stdout:Adding isodate 0.6.0 to easy-install.pth file 2019-11-25T10:26:30.745 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.745 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.746 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for gevent==1.4.0 2019-11-25T10:26:30.746 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: gevent 1.4.0 2019-11-25T10:26:30.746 INFO:teuthology.orchestra.run.smithi153.stdout:Adding gevent 1.4.0 to easy-install.pth file 2019-11-25T10:26:30.746 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.746 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.748 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for bunch==1.0.1 2019-11-25T10:26:30.748 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: bunch 1.0.1 2019-11-25T10:26:30.748 INFO:teuthology.orchestra.run.smithi153.stdout:Adding bunch 1.0.1 to easy-install.pth file 2019-11-25T10:26:30.748 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.748 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.749 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for PyYAML==5.1.2 2019-11-25T10:26:30.749 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: PyYAML 5.1.2 2019-11-25T10:26:30.749 INFO:teuthology.orchestra.run.smithi153.stdout:Adding PyYAML 5.1.2 to easy-install.pth file 2019-11-25T10:26:30.749 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.749 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.750 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for boto3==1.10.26 2019-11-25T10:26:30.751 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: boto3 1.10.26 2019-11-25T10:26:30.751 INFO:teuthology.orchestra.run.smithi153.stdout:Adding boto3 1.10.26 to easy-install.pth file 2019-11-25T10:26:30.751 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.751 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.751 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for boto==2.49.0 2019-11-25T10:26:30.751 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: boto 2.49.0 2019-11-25T10:26:30.752 INFO:teuthology.orchestra.run.smithi153.stdout:Adding boto 2.49.0 to easy-install.pth file 2019-11-25T10:26:30.752 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.752 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.753 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for six==1.13.0 2019-11-25T10:26:30.753 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: six 1.13.0 2019-11-25T10:26:30.754 INFO:teuthology.orchestra.run.smithi153.stdout:Adding six 1.13.0 to easy-install.pth file 2019-11-25T10:26:30.754 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.754 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.754 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for greenlet==0.4.15 2019-11-25T10:26:30.754 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: greenlet 0.4.15 2019-11-25T10:26:30.754 INFO:teuthology.orchestra.run.smithi153.stdout:Adding greenlet 0.4.15 to easy-install.pth file 2019-11-25T10:26:30.754 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.755 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.755 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for s3transfer==0.2.1 2019-11-25T10:26:30.755 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: s3transfer 0.2.1 2019-11-25T10:26:30.755 INFO:teuthology.orchestra.run.smithi153.stdout:Adding s3transfer 0.2.1 to easy-install.pth file 2019-11-25T10:26:30.755 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.755 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.758 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for botocore==1.13.26 2019-11-25T10:26:30.758 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: botocore 1.13.26 2019-11-25T10:26:30.758 INFO:teuthology.orchestra.run.smithi153.stdout:Adding botocore 1.13.26 to easy-install.pth file 2019-11-25T10:26:30.758 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.758 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.759 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for jmespath==0.9.4 2019-11-25T10:26:30.759 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: jmespath 0.9.4 2019-11-25T10:26:30.759 INFO:teuthology.orchestra.run.smithi153.stdout:Adding jmespath 0.9.4 to easy-install.pth file 2019-11-25T10:26:30.759 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.759 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.759 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for futures==3.3.0 2019-11-25T10:26:30.759 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: futures 3.3.0 2019-11-25T10:26:30.760 INFO:teuthology.orchestra.run.smithi153.stdout:Adding futures 3.3.0 to easy-install.pth file 2019-11-25T10:26:30.760 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.760 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.760 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for urllib3==1.25.7 2019-11-25T10:26:30.760 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: urllib3 1.25.7 2019-11-25T10:26:30.762 INFO:teuthology.orchestra.run.smithi153.stdout:Adding urllib3 1.25.7 to easy-install.pth file 2019-11-25T10:26:30.762 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.762 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.762 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for python-dateutil==2.8.0 2019-11-25T10:26:30.762 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: python-dateutil 2.8.0 2019-11-25T10:26:30.762 INFO:teuthology.orchestra.run.smithi153.stdout:Adding python-dateutil 2.8.0 to easy-install.pth file 2019-11-25T10:26:30.762 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.763 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.763 INFO:teuthology.orchestra.run.smithi153.stdout:Searching for docutils==0.15.2 2019-11-25T10:26:30.763 INFO:teuthology.orchestra.run.smithi153.stdout:Best match: docutils 0.15.2 2019-11-25T10:26:30.763 INFO:teuthology.orchestra.run.smithi153.stdout:Adding docutils 0.15.2 to easy-install.pth file 2019-11-25T10:26:30.763 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T10:26:30.763 INFO:teuthology.orchestra.run.smithi153.stdout:Using /home/ubuntu/cephtest/s3-tests/virtualenv/lib/python2.7/site-packages 2019-11-25T10:26:30.764 INFO:teuthology.orchestra.run.smithi153.stdout:Finished processing dependencies for s3tests==0.0.1 2019-11-25T10:26:30.769 INFO:teuthology.orchestra.run.smithi153:> cat > /home/ubuntu/cephtest/archive/s3-tests.client.0.conf 2019-11-25T10:26:30.819 INFO:tasks.s3tests:Configuring boto... 2019-11-25T10:26:30.820 INFO:teuthology.orchestra.run.smithi153:> cat > /home/ubuntu/cephtest/boto.cfg 2019-11-25T10:26:30.932 INFO:teuthology.orchestra.run.smithi153:s3 tests against rgw> S3TEST_CONF=/home/ubuntu/cephtest/archive/s3-tests.client.0.conf BOTO_CONFIG=/home/ubuntu/cephtest/boto.cfg REQUESTS_CA_BUNDLE=/etc/pki/tls/certs/ca-bundle.crt /home/ubuntu/cephtest/s3-tests/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/s3-tests -v -a '!fails_on_rgw,!lifecycle_expiration,!fails_strict_rfc2616,!fails_with_subdomain' 2019-11-25T10:26:35.449 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_none ... ok 2019-11-25T10:26:38.777 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_empty ... ok 2019-11-25T10:26:42.988 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_date_and_amz_date ... ok 2019-11-25T10:26:46.568 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_amz_date_and_no_date ... ok 2019-11-25T10:26:49.135 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_none ... ok 2019-11-25T10:26:51.261 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_contentlength_none ... ok 2019-11-25T10:26:56.669 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_acl_create_contentlength_none ... ok 2019-11-25T10:26:58.645 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_contentlength_none ... ok 2019-11-25T10:26:58.909 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_empty ... ok 2019-11-25T10:26:59.232 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_none ... ok 2019-11-25T10:27:01.102 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws2 ... ok 2019-11-25T10:27:02.751 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws2 ... ok 2019-11-25T10:27:03.592 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws2 ... ok 2019-11-25T10:27:04.589 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws2 ... ok 2019-11-25T10:27:04.679 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws2 ... ok 2019-11-25T10:27:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws2 ... ok 2019-11-25T10:27:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws4 ... SKIP 2019-11-25T10:27:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_contentlength_mismatch_below_aws4 ... SKIP 2019-11-25T10:27:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_incorrect_aws4 ... SKIP 2019-11-25T10:27:04.759 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_authorization_invalid_aws4 ... SKIP 2019-11-25T10:27:04.760 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_empty_aws4 ... SKIP 2019-11-25T10:27:04.760 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_ua_none_aws4 ... SKIP 2019-11-25T10:27:04.760 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_date_invalid_aws4 ... SKIP 2019-11-25T10:27:04.760 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_invalid_aws4 ... SKIP 2019-11-25T10:27:04.760 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_date_empty_aws4 ... SKIP 2019-11-25T10:27:04.767 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_empty_aws4 ... SKIP 2019-11-25T10:27:04.767 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_date_none_aws4 ... SKIP 2019-11-25T10:27:04.767 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_none_aws4 ... SKIP 2019-11-25T10:27:04.767 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_today_aws4 ... SKIP 2019-11-25T10:27:04.768 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_today_aws4 ... SKIP 2019-11-25T10:27:04.768 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_today_aws4 ... SKIP 2019-11-25T10:27:04.768 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_today_aws4 ... SKIP 2019-11-25T10:27:04.768 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_date_before_epoch_aws4 ... SKIP 2019-11-25T10:27:04.770 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-11-25T10:27:04.770 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_date_after_end_aws4 ... SKIP 2019-11-25T10:27:04.770 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_bad_amz_date_after_end_aws4 ... SKIP 2019-11-25T10:27:04.771 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_custom_header_aws4 ... SKIP 2019-11-25T10:27:04.771 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_object_create_missing_signed_header_aws4 ... SKIP 2019-11-25T10:27:04.771 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_authorization_invalid_aws4 ... SKIP 2019-11-25T10:27:04.771 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_empty_aws4 ... SKIP 2019-11-25T10:27:04.771 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_ua_none_aws4 ... SKIP 2019-11-25T10:27:04.771 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_invalid_aws4 ... SKIP 2019-11-25T10:27:04.772 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_invalid_aws4 ... SKIP 2019-11-25T10:27:04.772 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_empty_aws4 ... SKIP 2019-11-25T10:27:04.772 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_empty_aws4 ... SKIP 2019-11-25T10:27:04.772 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_none_aws4 ... SKIP 2019-11-25T10:27:04.772 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_none_aws4 ... SKIP 2019-11-25T10:27:04.772 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_today_aws4 ... SKIP 2019-11-25T10:27:04.772 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_today_aws4 ... SKIP 2019-11-25T10:27:04.773 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_after_today_aws4 ... SKIP 2019-11-25T10:27:04.773 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_after_today_aws4 ... SKIP 2019-11-25T10:27:04.773 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws4 ... SKIP 2019-11-25T10:27:04.773 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_headers.test_bucket_create_bad_amz_date_before_epoch_aws4 ... SKIP 2019-11-25T10:27:08.247 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_versioning_obj_read_not_exist_null ... ok 2019-11-25T10:27:08.248 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_lifecycle_transition ... SKIP 2019-11-25T10:27:08.248 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_lifecycle_transition_single_rule_multi_trans ... SKIP 2019-11-25T10:27:08.248 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_lifecycle_set_noncurrent_transition ... SKIP 2019-11-25T10:27:08.248 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_object_storage_class ... SKIP 2019-11-25T10:27:08.248 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_object_storage_class_multipart ... SKIP 2019-11-25T10:27:08.248 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_object_modify_storage_class ... SKIP 2019-11-25T10:27:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_object_modify_storage_class_multipart ... SKIP 2019-11-25T10:27:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_object_storage_class_copy ... SKIP 2019-11-25T10:27:08.249 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_object_storage_class_copy_multipart ... SKIP 2019-11-25T10:27:12.433 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_atomic_dual_conditional_write_1mb ... ok 2019-11-25T10:27:16.194 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_atomic_write_bucket_gone ... ok 2019-11-25T10:27:18.583 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_1 ... ok 2019-11-25T10:27:20.345 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_encryption_sse_c_multipart_invalid_chunks_2 ... ok 2019-11-25T10:27:23.643 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_bucket_policy_set_condition_operator_end_with_IfExists ... ok 2019-11-25T10:27:26.212 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_enc ... ok 2019-11-25T10:27:28.752 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3.test_bucket_policy_put_obj_request_obj_tag ... ok 2019-11-25T10:27:32.275 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.check_can_test_website ... SKIP 2019-11-25T10:27:32.275 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_nonexistant_bucket_rgw ... SKIP 2019-11-25T10:27:32.275 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_public_index ... SKIP 2019-11-25T10:27:32.275 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_public_index ... SKIP 2019-11-25T10:27:32.276 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty ... SKIP 2019-11-25T10:27:32.276 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty ... SKIP 2019-11-25T10:27:32.276 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index ... SKIP 2019-11-25T10:27:32.276 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index ... SKIP 2019-11-25T10:27:32.276 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_missingerrordoc ... SKIP 2019-11-25T10:27:32.276 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_missingerrordoc ... SKIP 2019-11-25T10:27:32.277 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_missingerrordoc ... SKIP 2019-11-25T10:27:32.277 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_missingerrordoc ... SKIP 2019-11-25T10:27:32.277 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_blockederrordoc ... SKIP 2019-11-25T10:27:32.277 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_blockederrordoc ... SKIP 2019-11-25T10:27:32.277 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_blockederrordoc ... SKIP 2019-11-25T10:27:32.277 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_blockederrordoc ... SKIP 2019-11-25T10:27:32.277 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_empty_gooderrordoc ... SKIP 2019-11-25T10:27:32.278 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_empty_gooderrordoc ... SKIP 2019-11-25T10:27:32.278 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_public_bucket_list_private_index_gooderrordoc ... SKIP 2019-11-25T10:27:32.278 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_private_bucket_list_private_index_gooderrordoc ... SKIP 2019-11-25T10:27:32.280 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_base ... SKIP 2019-11-25T10:27:32.280 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path ... SKIP 2019-11-25T10:27:32.280 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_bucket_private_redirectall_path_upgrade ... SKIP 2019-11-25T10:27:32.280 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_xredirect_nonwebsite ... SKIP 2019-11-25T10:27:32.281 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_relative ... SKIP 2019-11-25T10:27:32.281 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_xredirect_public_abs ... SKIP 2019-11-25T10:27:32.281 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_relative ... SKIP 2019-11-25T10:27:32.281 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_s3_website.test_website_xredirect_private_abs ... SKIP 2019-11-25T10:27:32.282 INFO:teuthology.orchestra.run.smithi153.stderr:SKIP 2019-11-25T10:27:32.290 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.functional.test_utils.test_generate ... ok 2019-11-25T10:28:21.441 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:13:26.018 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:28:21.445 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:13:26.023 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:28:21.454 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:13:26.058 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:28:21.454 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:13:26.058 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:28:21.458 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:13:26.062 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:28:21.458 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:13:26.062 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:28:21.660 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:13:26.266 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:28:21.670 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:13:26.275 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:28:21.685 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:13:26.267 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:29:37.379 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-11-25T10:29:37.380 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-11-25T10:29:37.380 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-11-25T10:29:37.380 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-11-25T10:29:37.380 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-11-25T10:29:37.380 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-11-25T10:29:37.380 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-11-25T10:29:37.766 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-11-25T10:29:37.969 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-11-25T10:29:38.284 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-11-25T10:29:38.307 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-11-25T10:29:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-11-25T10:29:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-11-25T10:29:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-11-25T10:29:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-11-25T10:29:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-11-25T10:29:38.327 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-11-25T10:29:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-11-25T10:29:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-11-25T10:29:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-11-25T10:29:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-11-25T10:29:38.328 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-11-25T10:29:38.331 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-11-25T10:29:38.332 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-11-25T10:29:38.332 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-11-25T10:29:38.332 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_null_child ... ok 2019-11-25T10:29:38.336 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-11-25T10:29:38.336 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-11-25T10:29:38.337 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-11-25T10:29:38.337 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-11-25T10:29:38.361 INFO:teuthology.orchestra.run.smithi153.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-11-25T10:29:38.385 INFO:teuthology.orchestra.run.smithi153.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-11-25T10:29:38.386 INFO:teuthology.orchestra.run.smithi153.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-11-25T10:29:38.550 INFO:teuthology.orchestra.run.smithi153.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-11-25T10:29:42.920 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_short ... ok 2019-11-25T10:29:58.476 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_bad ... ok 2019-11-25T10:30:01.901 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_empty ... ok 2019-11-25T10:30:08.512 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_none ... ok 2019-11-25T10:30:13.813 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_mismatch ... ok 2019-11-25T10:30:17.869 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_empty ... ok 2019-11-25T10:30:24.643 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_expect_none ... ok 2019-11-25T10:30:28.453 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contentlength_negative ... ok 2019-11-25T10:30:34.082 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_invalid ... ok 2019-11-25T10:30:38.031 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_empty ... ok 2019-11-25T10:30:42.566 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_contenttype_none ... ok 2019-11-25T10:30:45.998 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_put_bad_canned_acl ... ok 2019-11-25T10:30:48.054 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_mismatch ... ok 2019-11-25T10:30:50.818 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_expect_empty ... ok 2019-11-25T10:30:50.930 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_contentlength_negative ... ok 2019-11-25T10:30:54.215 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_md5_invalid_garbage_aws2 ... ok 2019-11-25T10:30:56.835 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:16:01.416 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:56.844 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:16:01.425 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:56.853 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:16:01.458 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:56.872 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:16:01.477 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:56.888 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:16:01.491 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:56.899 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:16:01.504 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:56.949 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:16:01.530 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:56.965 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:16:01.571 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:56.978 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:16:01.583 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:30:58.634 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_empty_aws2 ... ok 2019-11-25T10:31:03.108 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_ua_none_aws2 ... ok 2019-11-25T10:31:06.564 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_invalid_aws2 ... ok 2019-11-25T10:31:08.800 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_empty_aws2 ... ok 2019-11-25T10:31:11.762 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_today_aws2 ... ok 2019-11-25T10:31:13.902 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_before_epoch_aws2 ... ok 2019-11-25T10:31:16.204 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_object_create_bad_date_after_end_aws2 ... ok 2019-11-25T10:31:18.443 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_empty_aws2 ... ok 2019-11-25T10:31:20.874 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_ua_none_aws2 ... ok 2019-11-25T10:31:20.955 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_invalid_aws2 ... ok 2019-11-25T10:31:21.083 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_empty_aws2 ... ok 2019-11-25T10:31:21.347 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_today_aws2 ... ok 2019-11-25T10:31:21.721 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_after_today_aws2 ... ok 2019-11-25T10:31:21.933 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_headers.test_bucket_create_bad_date_before_epoch_aws2 ... ok 2019-11-25T10:31:26.164 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_empty ... ok 2019-11-25T10:31:33.241 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_distinct ... ok 2019-11-25T10:31:41.767 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_many ... ok 2019-11-25T10:31:50.197 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_many ... ok 2019-11-25T10:31:56.703 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_basic_key_count ... ok 2019-11-25T10:32:04.652 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_basic ... ok 2019-11-25T10:32:13.911 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_basic ... ok 2019-11-25T10:32:27.276 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix ... ok 2019-11-25T10:32:40.988 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix ... ok 2019-11-25T10:32:45.450 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_ends_with_delimiter ... ok 2019-11-25T10:32:51.496 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_ends_with_delimiter ... ok 2019-11-25T10:32:59.407 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_alt ... ok 2019-11-25T10:33:07.358 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_alt ... ok 2019-11-25T10:33:20.323 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_prefix_underscore ... ok 2019-11-25T10:33:32.503 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:18:37.084 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:32.519 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:18:37.101 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:32.530 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:18:37.135 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:32.531 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:18:37.135 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:32.536 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:18:37.141 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:32.540 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:18:37.145 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:32.545 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_prefix_underscore ... ok 2019-11-25T10:33:32.664 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:18:37.268 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:32.669 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:18:37.251 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:32.672 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:18:37.276 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:33:40.504 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_percentage ... ok 2019-11-25T10:33:47.741 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_percentage ... ok 2019-11-25T10:33:56.283 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_whitespace ... ok 2019-11-25T10:34:03.620 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_whitespace ... ok 2019-11-25T10:34:13.233 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_dot ... ok 2019-11-25T10:34:22.268 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_dot ... ok 2019-11-25T10:34:30.282 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_unreadable ... ok 2019-11-25T10:34:37.259 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_unreadable ... ok 2019-11-25T10:34:46.622 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_empty ... ok 2019-11-25T10:34:54.625 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_empty ... ok 2019-11-25T10:35:01.868 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_none ... ok 2019-11-25T10:35:10.119 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_none ... ok 2019-11-25T10:35:15.982 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_notempty ... ok 2019-11-25T10:35:22.409 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_defaultempty ... ok 2019-11-25T10:35:28.742 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_fetchowner_empty ... ok 2019-11-25T10:35:37.010 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_exist ... ok 2019-11-25T10:35:45.475 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_delimiter_not_exist ... ok 2019-11-25T10:38:44.440 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:23:49.022 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:38:44.450 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:23:49.032 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:38:44.463 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:23:49.067 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:38:44.466 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:23:49.070 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:38:44.470 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:23:49.074 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:38:44.471 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:23:49.075 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:38:44.663 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:23:49.245 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:38:44.702 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:23:49.306 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:38:44.710 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:23:49.314 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.546 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:24:23.128 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.583 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:24:23.188 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.588 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:24:23.192 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.601 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:24:23.183 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.620 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:24:23.224 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.623 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:24:23.227 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.646 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:24:23.250 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.648 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:24:23.230 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:39:18.684 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:24:23.288 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.738 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:29:00.319 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.742 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:29:00.324 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.753 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:29:00.356 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.757 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:29:00.361 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.759 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:29:00.363 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.764 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:29:00.368 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.930 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:29:00.534 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.934 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:29:00.515 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:43:55.937 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:29:00.541 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.239 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:32:49.821 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.293 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:32:49.874 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.293 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:32:49.897 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.297 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:32:49.901 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.313 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:32:49.917 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.314 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:32:49.918 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.406 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:32:49.988 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.416 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:32:50.019 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:47:45.429 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:32:50.032 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:06.959 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:34:11.540 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:06.969 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:34:11.550 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:06.977 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:34:11.580 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:06.977 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:34:11.581 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:06.979 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:34:11.583 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:06.987 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:34:11.591 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:07.040 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:34:11.644 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:07.060 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:34:11.664 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:49:07.071 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:34:11.653 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:18.923 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:39:23.504 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:18.930 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:39:23.511 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:18.938 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:39:23.541 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:18.939 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:39:23.543 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:18.942 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:39:23.546 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:18.943 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:39:23.547 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:19.117 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:39:23.720 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:19.151 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:39:23.755 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:54:19.169 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:39:23.750 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.332 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:41:11.913 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.334 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:41:11.916 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.387 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:41:11.990 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.387 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:41:11.991 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.391 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:41:11.995 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.391 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:41:11.995 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.412 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:41:11.993 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.500 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:41:12.103 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:07.509 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:41:12.113 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.511 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:41:13.092 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.512 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:41:13.094 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.524 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:41:13.106 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.697 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:41:13.300 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.699 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:41:13.303 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.705 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:41:13.309 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.708 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:41:13.312 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.717 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:41:13.321 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:08.728 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:41:13.331 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:56:31.508 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_delimiter_not_skip_special ... ok 2019-11-25T10:56:40.520 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_basic ... ok 2019-11-25T10:56:47.394 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_basic ... ok 2019-11-25T10:56:54.273 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_alt ... ok 2019-11-25T10:56:58.650 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_alt ... ok 2019-11-25T10:57:03.717 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_empty ... ok 2019-11-25T10:57:10.011 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_empty ... ok 2019-11-25T10:57:14.083 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_none ... ok 2019-11-25T10:57:19.831 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_none ... ok 2019-11-25T10:57:25.400 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_not_exist ... ok 2019-11-25T10:57:31.658 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_not_exist ... ok 2019-11-25T10:57:37.547 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_unreadable ... ok 2019-11-25T10:57:43.989 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_unreadable ... ok 2019-11-25T10:57:52.035 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_basic ... ok 2019-11-25T10:57:58.486 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_basic ... ok 2019-11-25T10:58:04.437 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_alt ... ok 2019-11-25T10:58:10.675 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_alt ... ok 2019-11-25T10:58:16.569 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_not_exist ... ok 2019-11-25T10:58:25.134 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_not_exist ... ok 2019-11-25T10:58:31.706 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_delimiter_not_exist ... ok 2019-11-25T10:58:41.061 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_delimiter_not_exist ... ok 2019-11-25T10:58:47.253 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-11-25T10:58:57.852 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_prefix_delimiter_prefix_delimiter_not_exist ... ok 2019-11-25T10:59:06.135 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_one ... ok 2019-11-25T10:59:16.725 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_one ... ok 2019-11-25T10:59:22.308 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_zero ... ok 2019-11-25T10:59:31.648 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_zero ... ok 2019-11-25T10:59:33.082 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:44:37.663 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:33.088 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:44:37.669 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:33.093 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:44:37.697 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:33.096 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:44:37.700 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:33.096 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:44:37.700 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:33.099 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:44:37.703 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:33.222 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:44:37.825 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:33.229 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:44:37.831 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:33.230 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:44:37.811 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T10:59:37.944 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_none ... ok 2019-11-25T10:59:48.336 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_maxkeys_none ... ok 2019-11-25T11:00:25.276 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_unordered ... ok 2019-11-25T11:01:01.709 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_unordered ... ok 2019-11-25T11:01:10.910 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_maxkeys_invalid ... ok 2019-11-25T11:01:19.129 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_none ... ok 2019-11-25T11:01:27.474 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_empty ... ok 2019-11-25T11:01:37.255 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_continuationtoken_empty ... ok 2019-11-25T11:01:45.583 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_unreadable ... ok 2019-11-25T11:01:54.461 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_unreadable ... ok 2019-11-25T11:02:03.110 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_not_in_list ... ok 2019-11-25T11:02:11.319 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_not_in_list ... ok 2019-11-25T11:02:20.169 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_marker_after_list ... ok 2019-11-25T11:02:29.542 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_startafter_after_list ... ok 2019-11-25T11:02:39.531 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data ... ok 2019-11-25T11:03:03.523 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_return_data_versioning ... ok 2019-11-25T11:03:08.389 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous ... ok 2019-11-25T11:03:14.770 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous ... ok 2019-11-25T11:03:17.330 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_objects_anonymous_fail ... ok 2019-11-25T11:03:19.893 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_listv2_objects_anonymous_fail ... ok 2019-11-25T11:03:20.252 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_notexist ... ok 2019-11-25T11:03:20.779 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_notexist ... ok 2019-11-25T11:03:21.146 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_notexist ... ok 2019-11-25T11:03:27.251 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_delete_nonempty ... ok 2019-11-25T11:03:34.768 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_concurrent_set_canned_acl ... ok 2019-11-25T11:03:34.882 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_write_to_nonexist_bucket ... ok 2019-11-25T11:03:42.135 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_delete ... ok 2019-11-25T11:03:45.130 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_read_notexist ... ok 2019-11-25T11:03:48.371 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_requestid_matches_header_on_error ... ok 2019-11-25T11:04:05.554 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multi_object_delete ... ok 2019-11-25T11:04:20.512 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multi_objectv2_delete ... ok 2019-11-25T11:04:24.956 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_head_zero_bytes ... ok 2019-11-25T11:04:27.916 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_write_check_etag ... ok 2019-11-25T11:04:28.973 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:49:33.555 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:28.984 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:49:33.565 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:29.020 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:49:33.623 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:29.045 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:49:33.648 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:29.047 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:49:33.651 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:29.064 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:49:33.667 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:29.201 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:49:33.802 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:29.207 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:49:33.810 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:29.249 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:49:33.822 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:31.602 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_write_cache_control ... ok 2019-11-25T11:04:37.172 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_write_expires ... ok 2019-11-25T11:04:43.292 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:49:47.872 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:43.300 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:49:47.881 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:43.309 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:49:47.912 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:43.314 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:49:47.917 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:43.315 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:49:47.918 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:43.317 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:49:47.921 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:43.345 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:49:47.947 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:43.352 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:49:47.931 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:43.369 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:49:47.972 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:04:45.219 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_write_read_update_read_delete ... ok 2019-11-25T11:04:50.095 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_good ... ok 2019-11-25T11:04:54.094 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_none_to_empty ... ok 2019-11-25T11:05:03.543 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_metadata_overwrite_to_empty ... ok 2019-11-25T11:05:08.447 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_set_get_unicode_metadata ... ok 2019-11-25T11:05:13.190 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_metadata_replaced_on_put ... ok 2019-11-25T11:05:17.909 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_write_file ... ok 2019-11-25T11:05:22.675 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_anonymous_request ... ok 2019-11-25T11:05:27.478 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request ... ok 2019-11-25T11:05:32.402 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_no_content_type ... ok 2019-11-25T11:05:36.462 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_authenticated_request_bad_access_key ... ok 2019-11-25T11:05:39.840 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_success_code ... ok 2019-11-25T11:05:43.409 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_invalid_success_code ... ok 2019-11-25T11:05:51.279 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_larger_than_chunk ... ok 2019-11-25T11:05:56.567 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_set_key_from_filename ... ok 2019-11-25T11:06:00.465 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_ignored_header ... ok 2019-11-25T11:06:04.282 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_case_insensitive_condition_fields ... ok 2019-11-25T11:06:10.066 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_escaped_field_values ... ok 2019-11-25T11:06:16.089 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_success_redirect_action ... ok 2019-11-25T11:06:19.115 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_signature ... ok 2019-11-25T11:06:23.658 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_access_key ... ok 2019-11-25T11:06:29.083 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_date_format ... ok 2019-11-25T11:06:31.921 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_no_key_specified ... ok 2019-11-25T11:06:35.043 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_signature ... ok 2019-11-25T11:06:38.576 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_policy_condition ... ok 2019-11-25T11:06:42.642 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_user_specified_header ... ok 2019-11-25T11:06:45.161 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_request_missing_policy_specified_field ... ok 2019-11-25T11:06:46.358 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_condition_is_case_sensitive ... ok 2019-11-25T11:06:50.054 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_expires_is_case_sensitive ... ok 2019-11-25T11:06:53.086 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_expired_policy ... ok 2019-11-25T11:06:55.521 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_request_field_value ... ok 2019-11-25T11:06:57.296 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_expires_condition ... ok 2019-11-25T11:07:00.955 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_conditions_list ... ok 2019-11-25T11:07:03.670 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_limit_exceeded ... ok 2019-11-25T11:07:05.951 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_missing_content_length_argument ... ok 2019-11-25T11:07:08.531 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_invalid_content_length_argument ... ok 2019-11-25T11:07:11.252 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_upload_size_below_minimum ... ok 2019-11-25T11:07:14.673 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_empty_conditions ... ok 2019-11-25T11:07:19.251 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_good ... ok 2019-11-25T11:07:22.353 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmatch_failed ... ok 2019-11-25T11:07:26.907 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_good ... ok 2019-11-25T11:07:31.450 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifnonematch_failed ... ok 2019-11-25T11:07:35.768 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_good ... ok 2019-11-25T11:07:41.726 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifmodifiedsince_failed ... ok 2019-11-25T11:07:45.843 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_good ... ok 2019-11-25T11:07:49.984 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_object_ifunmodifiedsince_failed ... ok 2019-11-25T11:07:55.268 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_good ... ok 2019-11-25T11:08:00.278 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_failed ... ok 2019-11-25T11:08:06.135 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_overwrite_existed_good ... ok 2019-11-25T11:08:09.110 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifmatch_nonexisted_failed ... ok 2019-11-25T11:08:14.961 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_good ... ok 2019-11-25T11:08:20.258 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_failed ... ok 2019-11-25T11:08:24.404 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_nonexisted_good ... ok 2019-11-25T11:08:28.868 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_object_ifnonmatch_overwrite_existed_failed ... ok 2019-11-25T11:08:32.511 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get ... ok 2019-11-25T11:08:44.013 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_gone ... ok 2019-11-25T11:08:55.183 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_delete_key_bucket_gone ... ok 2019-11-25T11:09:01.294 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_gone ... ok 2019-11-25T11:09:03.815 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_head ... ok 2019-11-25T11:09:15.506 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_head_extended ... ok 2019-11-25T11:09:19.815 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_bucket_acl ... ok 2019-11-25T11:09:25.270 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_object_acl ... ok 2019-11-25T11:09:29.043 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated ... ok 2019-11-25T11:09:33.185 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_response_headers ... ok 2019-11-25T11:09:38.461 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_acl ... ok 2019-11-25T11:09:43.271 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_acl ... ok 2019-11-25T11:09:53.441 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_bucket_gone ... ok 2019-11-25T11:09:55.262 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:54:59.842 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:55.270 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:54:59.852 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:55.279 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:54:59.882 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:55.288 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:54:59.890 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:55.288 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:54:59.890 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:55.291 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:54:59.894 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:55.457 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:55:00.038 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:55.486 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:55:00.088 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:55.496 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:55:00.099 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:09:59.507 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_authenticated_object_gone ... ok 2019-11-25T11:10:04.354 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_not_expired ... ok 2019-11-25T11:10:07.565 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_range_zero ... ok 2019-11-25T11:10:10.393 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_max_range ... ok 2019-11-25T11:10:14.574 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_get_x_amz_expires_out_positive_range ... ok 2019-11-25T11:10:17.738 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put ... ok 2019-11-25T11:10:23.261 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_anon_put_write_access ... ok 2019-11-25T11:10:28.909 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_put_authenticated ... ok 2019-11-25T11:10:31.908 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_raw_put_authenticated_expired ... ok 2019-11-25T11:10:32.597 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_one ... ok 2019-11-25T11:10:33.417 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_short_two ... ok 2019-11-25T11:10:34.135 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_bad_ip ... ok 2019-11-25T11:10:34.626 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_dns_underscore ... ok 2019-11-25T11:10:41.141 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists ... ok 2019-11-25T11:10:45.351 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_get_location ... ok 2019-11-25T11:10:50.003 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_exists_nonowner ... ok 2019-11-25T11:10:52.443 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_default ... ok 2019-11-25T11:10:56.472 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_during_create ... ok 2019-11-25T11:11:00.636 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned ... ok 2019-11-25T11:11:04.492 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_publicreadwrite ... ok 2019-11-25T11:11:07.670 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_authenticatedread ... ok 2019-11-25T11:11:12.270 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_default ... ok 2019-11-25T11:11:17.160 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_during_create ... ok 2019-11-25T11:11:23.402 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned ... ok 2019-11-25T11:11:28.311 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_publicreadwrite ... ok 2019-11-25T11:11:33.457 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_authenticatedread ... ok 2019-11-25T11:11:38.753 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerread ... ok 2019-11-25T11:11:46.343 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_canned_bucketownerfullcontrol ... ok 2019-11-25T11:11:53.455 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_owner ... ok 2019-11-25T11:11:59.728 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_full_control_verify_attributes ... ok 2019-11-25T11:12:05.545 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_canned_private_to_private ... ok 2019-11-25T11:12:10.863 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl ... ok 2019-11-25T11:12:16.460 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_write ... ok 2019-11-25T11:12:22.034 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_writeacp ... ok 2019-11-25T11:12:26.968 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_read ... ok 2019-11-25T11:12:34.508 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_acl_readacp ... ok 2019-11-25T11:12:40.832 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_fullcontrol ... ok 2019-11-25T11:12:45.244 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_read ... ok 2019-11-25T11:12:49.530 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_readacp ... ok 2019-11-25T11:12:50.913 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:57:55.488 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:50.914 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:57:55.516 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:50.914 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:57:55.496 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:50.953 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:57:55.555 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:50.953 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:57:55.556 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:50.957 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:57:55.560 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:51.029 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:01:57:55.610 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:51.031 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:01:57:55.634 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:51.070 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:01:57:55.673 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:12:54.833 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_write ... ok 2019-11-25T11:12:59.642 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_userid_writeacp ... ok 2019-11-25T11:13:03.373 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_nonexist_user ... ok 2019-11-25T11:13:10.686 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_no_grants ... ok 2019-11-25T11:13:15.992 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_header_acl_grants ... ok 2019-11-25T11:13:23.971 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_header_acl_grants ... ok 2019-11-25T11:13:27.621 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email ... ok 2019-11-25T11:13:30.492 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_grant_email_notexist ... ok 2019-11-25T11:13:37.516 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_acl_revoke_all ... ok 2019-11-25T11:13:49.258 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_private ... ok 2019-11-25T11:13:57.894 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_private ... ok 2019-11-25T11:14:06.246 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicread ... ok 2019-11-25T11:14:15.127 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicread ... ok 2019-11-25T11:14:23.196 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_object_publicreadwrite ... ok 2019-11-25T11:14:33.070 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_private_objectv2_publicreadwrite ... ok 2019-11-25T11:14:45.057 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_private ... ok 2019-11-25T11:14:54.171 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicread ... ok 2019-11-25T11:15:05.653 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicread_object_publicreadwrite ... ok 2019-11-25T11:15:05.729 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:00:10.309 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:05.737 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:00:10.318 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:05.744 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:00:10.347 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:05.748 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:00:10.351 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:05.748 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:00:10.351 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:05.753 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:00:10.356 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:05.850 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:00:10.431 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:05.855 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:00:10.458 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:05.890 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:00:10.492 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:15:17.372 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_private ... ok 2019-11-25T11:15:30.963 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicread ... ok 2019-11-25T11:15:44.265 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_access_bucket_publicreadwrite_object_publicreadwrite ... ok 2019-11-25T11:16:00.252 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_buckets_create_then_list ... ok 2019-11-25T11:16:01.579 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_anonymous ... ok 2019-11-25T11:16:02.346 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_invalid_auth ... ok 2019-11-25T11:16:03.325 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_list_buckets_bad_auth ... ok 2019-11-25T11:16:12.212 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_alpha ... ok 2019-11-25T11:16:22.962 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_starts_digit ... ok 2019-11-25T11:16:24.356 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_period ... ok 2019-11-25T11:16:27.591 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_naming_good_contains_hyphen ... ok 2019-11-25T11:16:37.488 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_recreate_not_overriding ... ok 2019-11-25T11:17:11.768 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_create_special_key_names ... ok 2019-11-25T11:17:21.189 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_list_special_prefix ... ok 2019-11-25T11:17:27.644 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_zero_size ... ok 2019-11-25T11:17:33.869 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_same_bucket ... ok 2019-11-25T11:17:40.236 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_verify_contenttype ... ok 2019-11-25T11:17:44.410 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself ... ok 2019-11-25T11:17:49.351 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_to_itself_with_metadata ... ok 2019-11-25T11:17:57.933 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_diff_bucket ... ok 2019-11-25T11:18:02.125 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_bucket ... ok 2019-11-25T11:18:11.617 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_not_owned_object_bucket ... ok 2019-11-25T11:18:19.647 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_canned_acl ... ok 2019-11-25T11:18:30.127 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_retaining_metadata ... ok 2019-11-25T11:18:41.400 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_replacing_metadata ... ok 2019-11-25T11:18:45.099 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_bucket_not_found ... ok 2019-11-25T11:18:47.562 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_key_not_found ... ok 2019-11-25T11:19:26.019 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_bucket ... ok 2019-11-25T11:19:42.285 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioned_url_encoding ... ok 2019-11-25T11:20:15.810 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:05:20.391 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:20:15.820 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:05:20.402 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:20:15.839 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:05:20.442 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:20:15.840 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:05:20.443 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:20:15.843 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:05:20.447 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:20:15.844 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:05:20.448 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:20:16.043 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:05:20.646 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:20:16.046 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:05:20.649 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:20:16.132 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:05:20.713 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.630 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:06:17.211 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.672 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:06:17.253 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.684 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:06:17.288 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.700 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:06:17.304 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.702 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:06:17.305 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.712 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:06:17.293 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.724 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:06:17.327 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.748 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:06:17.348 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:12.766 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:06:17.370 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:21:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_copy_versioning_multipart_upload ... ok 2019-11-25T11:21:30.251 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_empty ... ok 2019-11-25T11:21:39.641 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_small ... ok 2019-11-25T11:22:00.807 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:22:00.766+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:22:01.767 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_small ... ok 2019-11-25T11:22:07.928 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_invalid_range ... ok 2019-11-25T11:22:24.017 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_without_range ... ok 2019-11-25T11:22:44.905 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:22:44.879+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:22:58.505 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:22:58.490+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:23:28.198 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:23:28.166+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:23:28.811 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_special_names ... ok 2019-11-25T11:24:02.851 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:24:02.696+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:24:34.132 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:24:34.051+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:24:37.613 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload ... ok 2019-11-25T11:25:27.969 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:10:32.551 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:27.980 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:10:32.558 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:27.983 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:10:32.587 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:27.989 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:10:32.593 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:27.990 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:10:32.594 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:27.991 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:10:32.595 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:28.143 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:10:32.724 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:28.152 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:10:32.756 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:28.156 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:10:32.760 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:25:28.277 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:25:28.230+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:25:45.179 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:25:45.119+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:25:48.254 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:25:48.181+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:26:10.199 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:26:10.174+0000 40a5f700 -1 handle_data failed with exit code -5 2019-11-25T11:26:14.866 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:26:14.816+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:26:16.297 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_versioned ... ok 2019-11-25T11:26:16.350 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:26:16.302+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:26:49.811 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:26:49.802+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:27:09.641 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:27:09.541+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:27:50.203 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:27:50.090+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:28:20.032 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:28:19.933+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:29:34.678 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:14:39.259 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:29:34.715 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:14:39.319 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:29:34.715 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:14:39.319 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:29:34.720 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:14:39.324 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:29:34.731 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:14:39.335 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:29:34.734 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:14:39.315 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:29:34.820 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:14:39.424 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:29:34.847 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:14:39.450 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:29:34.876 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:14:39.457 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:19.284 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:30:19.246+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:30:36.198 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:15:40.779 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:36.209 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:15:40.813 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:36.212 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:15:40.816 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:36.214 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:15:40.818 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:36.215 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:15:40.819 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:36.229 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:15:40.811 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:36.276 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:15:40.880 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:36.307 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:15:40.911 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:36.333 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:15:40.915 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:30:46.902 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:30:46.784+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:31:40.225 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:31:40.186+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:32:06.671 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:32:06.590+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:32:09.698 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_resend_part ... ok 2019-11-25T11:33:22.541 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_multiple_sizes ... ok 2019-11-25T11:33:45.697 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:33:45.685+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:34:00.089 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:34:00.056+0000 3fa5d700 -1 handle_data failed with exit code -5 2019-11-25T11:34:15.235 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:34:15.217+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:34:35.136 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:34:35.058+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:34:55.364 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:34:55.284+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:35:14.723 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T11:35:14.696+0000 4025e700 -1 handle_data failed with exit code -5 2019-11-25T11:35:16.390 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_copy_multiple_sizes ... ok 2019-11-25T11:35:43.385 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_size_too_small ... ok 2019-11-25T11:35:48.349 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:20:52.930 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:35:48.355 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:20:52.937 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:35:48.364 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:20:52.968 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:35:48.366 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:20:52.970 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:35:48.370 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:20:52.974 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:35:48.380 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:20:52.984 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:35:48.653 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:20:53.235 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:35:48.679 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:20:53.283 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:35:48.692 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:20:53.296 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:03.851 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_contents ... ok 2019-11-25T11:36:16.410 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:21:20.992 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:16.412 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:21:20.994 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:16.454 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:21:21.036 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:16.504 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:21:21.108 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:16.506 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:21:21.109 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:16.506 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:21:21.110 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:16.512 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:21:21.116 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:16.536 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:21:21.140 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:16.543 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:21:21.147 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:36:21.684 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_overwrite_existing_object ... ok 2019-11-25T11:36:37.594 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload ... ok 2019-11-25T11:36:41.265 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_abort_multipart_upload_not_found ... ok 2019-11-25T11:37:05.809 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_list_multipart_upload ... ok 2019-11-25T11:37:12.358 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_missing_part ... ok 2019-11-25T11:37:18.157 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_upload_incorrect_etag ... ok 2019-11-25T11:37:22.528 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_100_continue ... ok 2019-11-25T11:37:26.859 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_set_cors ... ok 2019-11-25T11:37:51.955 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_response ... ok 2019-11-25T11:38:01.108 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_cors_origin_wildcard ... ok 2019-11-25T11:38:01.708 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:23:06.289 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:01.712 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:23:06.294 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:01.747 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:23:06.349 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:01.758 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:23:06.362 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:01.769 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:23:06.373 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:01.770 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:23:06.351 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:01.777 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:23:06.381 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:01.844 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:23:06.448 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:01.849 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:23:06.453 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:38:06.455 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_cors_header_option ... ok 2019-11-25T11:38:14.237 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_1mb ... ok 2019-11-25T11:38:25.110 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_4mb ... ok 2019-11-25T11:38:36.126 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_read_8mb ... ok 2019-11-25T11:38:45.648 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_1mb ... ok 2019-11-25T11:38:59.493 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_4mb ... ok 2019-11-25T11:39:12.884 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_write_8mb ... ok 2019-11-25T11:39:28.480 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_1mb ... ok 2019-11-25T11:39:44.688 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_4mb ... ok 2019-11-25T11:40:07.277 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_dual_write_8mb ... ok 2019-11-25T11:40:17.689 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_conditional_write_1mb ... ok 2019-11-25T11:40:25.194 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_atomic_multipart_upload_write ... ok 2019-11-25T11:40:47.889 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_multipart_resend_first_finishes_last ... ok 2019-11-25T11:40:52.002 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_response_code ... ok 2019-11-25T11:40:57.408 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:26:01.989 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:40:57.418 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:26:02.000 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:40:57.420 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:26:02.023 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:40:57.421 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:26:02.024 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:40:57.428 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:26:02.031 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:40:57.429 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:26:02.033 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:40:57.559 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:26:02.162 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:40:57.575 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:26:02.179 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:40:57.585 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:26:02.167 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:41:00.443 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_ranged_big_request_response_code ... ok 2019-11-25T11:41:04.941 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_skip_leading_bytes_response_code ... ok 2019-11-25T11:41:10.151 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_return_trailing_bytes_response_code ... ok 2019-11-25T11:41:15.623 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_invalid_range ... ok 2019-11-25T11:41:20.320 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_ranged_request_empty_object ... ok 2019-11-25T11:41:28.666 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_create_suspend ... ok 2019-11-25T11:46:10.602 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:31:15.184 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:10.610 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:31:15.192 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:10.612 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:31:15.215 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:10.616 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:31:15.219 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:10.617 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:31:15.221 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:10.621 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:31:15.224 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:10.800 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:31:15.382 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:10.806 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:31:15.409 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:10.822 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:31:15.425 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.460 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:31:28.042 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.467 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:31:28.049 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.492 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:31:28.074 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.497 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:31:28.101 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.510 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:31:28.113 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.522 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:31:28.125 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.524 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:31:28.128 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.540 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:31:28.144 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:23.546 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:31:28.149 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:46:49.186 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove ... ok 2019-11-25T11:47:35.407 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_read_remove_head ... ok 2019-11-25T11:47:44.790 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_removal ... ok 2019-11-25T11:48:04.767 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite ... ok 2019-11-25T11:48:19.078 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_plain_null_version_overwrite_suspended ... ok 2019-11-25T11:50:25.034 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_suspend_versions ... ok 2019-11-25T11:51:22.772 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:36:27.354 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:51:22.778 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:36:27.360 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:51:22.783 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:36:27.387 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:51:22.786 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:36:27.390 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:51:22.791 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:36:27.394 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:51:22.796 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:36:27.400 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:51:22.969 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:36:27.572 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:51:23.027 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:36:27.610 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:51:23.030 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:36:27.634 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:52:29.941 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_all ... ok 2019-11-25T11:54:45.238 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:39:49.820 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:54:45.245 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:39:49.828 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:54:45.281 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:39:49.885 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:54:45.286 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:39:49.890 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:54:45.333 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:39:49.937 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:54:45.336 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:39:49.940 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:54:45.361 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:39:49.943 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:54:45.400 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:39:50.004 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:54:45.462 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:39:50.066 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:34.974 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:41:39.556 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:34.983 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:41:39.587 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:34.983 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:41:39.565 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:34.986 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:41:39.591 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:34.992 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:41:39.594 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:34.992 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:41:39.595 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:35.074 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:41:39.678 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:35.101 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:41:39.705 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T11:56:35.130 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:41:39.712 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:00:06.703 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_versions_remove_special_names ... ok 2019-11-25T12:01:46.207 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:46:50.789 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:46.224 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:46:50.807 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:46.230 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:46:50.835 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:46.232 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:46:50.837 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:46.237 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:46:50.841 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:46.237 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:46:50.841 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:46.458 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:46:51.061 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:46.462 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:46:51.063 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:46.484 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:46:51.066 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:01:59.957 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_create_overwrite_multipart ... ok 2019-11-25T12:02:46.348 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_obj_list_marker ... ok 2019-11-25T12:03:07.244 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:48:11.826 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:07.292 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:48:11.896 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:07.294 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:48:11.898 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:07.301 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:48:11.883 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:07.313 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:48:11.917 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:07.318 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:48:11.922 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:07.347 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:48:11.951 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:07.354 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:48:11.937 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:07.386 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:48:11.991 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:03:29.206 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_copy_obj_version ... ok 2019-11-25T12:03:54.064 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete ... ok 2019-11-25T12:04:30.130 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker ... ok 2019-11-25T12:04:37.202 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_multi_object_delete_with_marker_create ... ok 2019-11-25T12:05:03.336 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl ... ok 2019-11-25T12:05:21.601 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioned_object_acl_no_version_specified ... ok 2019-11-25T12:06:53.914 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_concurrent_remove ... ok 2019-11-25T12:06:57.338 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:52:01.920 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:06:57.349 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:52:01.931 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:06:57.350 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:52:01.954 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:06:57.355 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:52:01.959 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:06:57.356 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:52:01.961 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:06:57.360 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:52:01.964 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:06:57.509 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:52:02.111 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:06:57.509 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:52:02.111 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:06:57.526 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:52:02.108 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:07:23.223 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioned_concurrent_object_create_and_remove ... ok 2019-11-25T12:07:28.445 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set ... ok 2019-11-25T12:07:33.652 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get ... ok 2019-11-25T12:07:37.153 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_get_no_id ... ok 2019-11-25T12:07:38.667 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_id_too_long ... ok 2019-11-25T12:07:41.170 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_same_id ... ok 2019-11-25T12:07:44.774 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_invalid_status ... ok 2019-11-25T12:07:49.369 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_date ... ok 2019-11-25T12:07:51.646 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_invalid_date ... ok 2019-11-25T12:07:56.877 INFO:teuthology.orchestra.run.smithi153.stderr:Check for valid x-amz-expiration header after PUT ... ok 2019-11-25T12:08:01.820 INFO:teuthology.orchestra.run.smithi153.stderr:Check for valid x-amz-expiration header on HEAD request ... ok 2019-11-25T12:08:07.509 INFO:teuthology.orchestra.run.smithi153.stderr:Check for valid x-amz-expiration header on GET request ... ok 2019-11-25T12:08:15.540 INFO:teuthology.orchestra.run.smithi153.stderr:Check x-amz-expiration-header object tags interaction ... ok 2019-11-25T12:08:22.206 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_noncurrent ... ok 2019-11-25T12:08:31.131 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_deletemarker ... ok 2019-11-25T12:08:36.758 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_filter ... ok 2019-11-25T12:08:41.238 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_empty_filter ... ok 2019-11-25T12:08:45.894 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_lifecycle_set_multipart ... ok 2019-11-25T12:08:50.427 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1b ... ok 2019-11-25T12:08:55.215 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1kb ... ok 2019-11-25T12:08:59.544 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_1MB ... ok 2019-11-25T12:09:03.530 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encrypted_transfer_13b ... ok 2019-11-25T12:09:08.526 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_method_head ... ok 2019-11-25T12:09:12.047 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_present ... ok 2019-11-25T12:09:16.003 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_other_key ... ok 2019-11-25T12:09:18.443 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_invalid_md5 ... ok 2019-11-25T12:09:20.819 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_md5 ... ok 2019-11-25T12:09:23.991 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_no_key ... ok 2019-11-25T12:09:26.789 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_key_no_sse_c ... ok 2019-11-25T12:10:58.335 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_upload ... ok 2019-11-25T12:11:34.168 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:56:38.750 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:34.182 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:56:38.764 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:34.234 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:56:38.838 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:34.254 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:56:38.859 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:34.263 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:56:38.868 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:34.269 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:56:38.873 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:34.351 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:56:38.933 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:34.410 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:56:39.014 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:34.418 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:56:39.021 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:11:37.097 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_multipart_bad_download ... ok 2019-11-25T12:11:41.239 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_encryption_sse_c_post_object_authenticated_request ... ok 2019-11-25T12:11:48.142 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_method_head ... ok 2019-11-25T12:11:52.006 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_present ... ok 2019-11-25T12:11:54.668 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_no_key ... ok 2019-11-25T12:11:57.319 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_not_declared ... ok 2019-11-25T12:12:09.375 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:57:13.956 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:12:09.388 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:57:13.970 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:12:09.393 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:57:13.998 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:12:09.399 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:57:14.003 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:12:09.401 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:57:14.005 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:12:09.404 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:57:14.009 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:12:09.443 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:02:57:14.047 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:12:09.444 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:02:57:14.048 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:12:09.473 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:02:57:14.055 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:13:25.672 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_upload ... ok 2019-11-25T12:13:55.623 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_1 ... ok 2019-11-25T12:14:32.316 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_multipart_invalid_chunks_2 ... ok 2019-11-25T12:14:36.690 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_post_object_authenticated_request ... ok 2019-11-25T12:14:41.446 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1b ... ok 2019-11-25T12:14:46.329 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1kb ... ok 2019-11-25T12:14:49.666 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_1MB ... ok 2019-11-25T12:14:55.565 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_transfer_13b ... ok 2019-11-25T12:14:59.815 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_sse_kms_read_declare ... ok 2019-11-25T12:15:05.505 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy ... ok 2019-11-25T12:15:12.116 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy ... ok 2019-11-25T12:15:22.023 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_acl ... ok 2019-11-25T12:15:29.839 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_acl ... ok 2019-11-25T12:15:39.951 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_another_bucket ... ok 2019-11-25T12:15:53.049 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucketv2_policy_another_bucket ... ok 2019-11-25T12:16:01.249 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_obj_tagging ... ok 2019-11-25T12:16:08.338 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_obj_head_tagging ... ok 2019-11-25T12:16:16.542 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_max_tags ... ok 2019-11-25T12:16:22.745 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_excess_tags ... ok 2019-11-25T12:16:29.232 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_max_kvsize_tags ... ok 2019-11-25T12:16:38.669 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_excess_key_tags ... ok 2019-11-25T12:16:44.577 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_excess_val_tags ... ok 2019-11-25T12:16:54.326 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_modify_tags ... ok 2019-11-25T12:17:00.948 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_delete_tags ... ok 2019-11-25T12:17:04.054 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_anonymous_request ... ok 2019-11-25T12:17:07.174 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_post_object_tags_authenticated_request ... ok 2019-11-25T12:17:11.804 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_obj_with_tags ... ok 2019-11-25T12:17:19.660 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_get_tags_acl_public ... ok 2019-11-25T12:17:20.588 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:02:25.169 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:20.618 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:02:25.200 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:20.640 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:02:25.244 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:20.646 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:02:25.251 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:20.670 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:02:25.275 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:20.679 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:02:25.283 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:20.852 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:02:25.456 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:20.879 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:02:25.483 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:20.882 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:02:25.463 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.170 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:02:26.774 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.172 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:02:26.776 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.175 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:02:26.779 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.177 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:02:26.781 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.200 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:02:26.804 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.202 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:02:26.806 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.239 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:02:26.821 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.241 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:02:26.823 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:22.261 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:02:26.842 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:17:26.909 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_put_tags_acl_public ... ok 2019-11-25T12:17:36.406 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_delete_tags_obj_public ... ok 2019-11-25T12:17:53.127 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_atomic_upload_return_version_id ... ok 2019-11-25T12:19:33.072 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_versioning_bucket_multipart_upload_return_version_id ... ok 2019-11-25T12:19:47.571 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_existing_tag ... ok 2019-11-25T12:19:56.211 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:05:00.793 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:19:56.216 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:05:00.797 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:19:56.250 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:05:00.854 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:19:56.257 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:05:00.862 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:19:56.258 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:05:00.863 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:19:56.261 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:05:00.866 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:19:56.302 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:05:00.883 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:19:56.337 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:05:00.942 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:19:56.388 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:05:00.992 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:20:00.107 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_tagging_existing_tag ... ok 2019-11-25T12:20:13.314 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_tagging_existing_tag ... ok 2019-11-25T12:20:28.836 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source ... ok 2019-11-25T12:20:42.036 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_copy_source_meta ... ok 2019-11-25T12:20:47.106 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_acl ... ok 2019-11-25T12:20:59.344 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_put_obj_grant ... ok 2019-11-25T12:21:13.472 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_bucket_policy_get_obj_acl_existing_tag ... ok 2019-11-25T12:21:18.176 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock ... ok 2019-11-25T12:21:21.238 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_bucket ... ok 2019-11-25T12:21:25.360 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_with_days_and_years ... ok 2019-11-25T12:21:28.245 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_days ... ok 2019-11-25T12:21:31.456 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_years ... ok 2019-11-25T12:21:34.627 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_lock_invalid_status ... ok 2019-11-25T12:21:37.891 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_suspend_versioning ... ok 2019-11-25T12:21:42.191 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock ... ok 2019-11-25T12:21:45.338 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_lock_invalid_bucket ... ok 2019-11-25T12:21:58.200 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention ... ok 2019-11-25T12:22:02.603 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_bucket ... ok 2019-11-25T12:22:09.343 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_invalid_mode ... ok 2019-11-25T12:22:26.257 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention ... ok 2019-11-25T12:22:31.182 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_retention_invalid_bucket ... ok 2019-11-25T12:22:31.814 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:07:36.396 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:31.820 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:07:36.401 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:31.831 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:07:36.436 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:31.834 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:07:36.438 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:31.836 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:07:36.440 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:31.842 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:07:36.446 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:31.922 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:07:36.503 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:31.940 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:07:36.544 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:31.963 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:07:36.567 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:22:46.700 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_versionid ... ok 2019-11-25T12:23:01.067 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_override_default_retention ... ok 2019-11-25T12:23:16.154 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_increase_period ... ok 2019-11-25T12:23:32.558 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period ... ok 2019-11-25T12:23:51.534 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_obj_retention_shorten_period_bypass ... ok 2019-11-25T12:24:05.351 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_retention ... ok 2019-11-25T12:24:14.072 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold ... ok 2019-11-25T12:24:17.999 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_bucket ... ok 2019-11-25T12:24:25.847 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_put_legal_hold_invalid_status ... ok 2019-11-25T12:24:38.019 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold ... ok 2019-11-25T12:24:41.504 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_legal_hold_invalid_bucket ... ok 2019-11-25T12:24:51.279 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_on ... ok 2019-11-25T12:25:04.405 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_delete_object_with_legal_hold_off ... ok 2019-11-25T12:25:26.382 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_get_obj_metadata ... ok 2019-11-25T12:25:43.691 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_s3.test_object_lock_uploading_obj ... ok 2019-11-25T12:25:43.708 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.functional.test_utils.test_generate ... ok 2019-11-25T12:27:45.812 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:12:50.393 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:27:45.817 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:12:50.398 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:27:45.827 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:12:50.431 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:27:45.828 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:12:50.432 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:27:45.831 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:12:50.435 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:27:45.832 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:12:50.436 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:27:46.024 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:12:50.628 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:27:46.030 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:12:50.634 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:27:46.047 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:12:50.628 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:17.942 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:13:22.523 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:17.989 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:13:22.570 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:18.003 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:13:22.607 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:18.006 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:13:22.610 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:18.022 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:13:22.603 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:18.025 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:13:22.629 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:18.033 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:13:22.637 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:18.040 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:13:22.644 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:28:18.068 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:13:22.673 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.155 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:18:02.736 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.167 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:18:02.748 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.177 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:18:02.781 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.182 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:18:02.786 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.193 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:18:02.795 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.196 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:18:02.800 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.352 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:18:02.933 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.389 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:18:02.992 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:32:58.395 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:18:02.999 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.783 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:21:44.365 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.812 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:21:44.416 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.824 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:21:44.406 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.825 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:21:44.429 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.855 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:21:44.459 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.858 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:21:44.462 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.921 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:21:44.525 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.929 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:21:44.510 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:36:39.954 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:21:44.557 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.357 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:23:14.938 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.390 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:23:14.994 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.394 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:23:14.997 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.411 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:23:14.992 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.422 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:23:15.026 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.425 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:23:15.029 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.497 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:23:15.078 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.502 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:23:15.106 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:38:10.507 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:23:15.111 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.459 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:28:27.040 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.491 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:28:27.072 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.494 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:28:27.098 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.502 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:28:27.106 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.523 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:28:27.128 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.531 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:28:27.135 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.671 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:28:27.275 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.679 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:28:27.260 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:43:22.750 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:28:27.355 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.693 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:30:06.274 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.708 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:30:06.288 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.740 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:30:06.344 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.743 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:30:06.346 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.767 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:30:06.372 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.775 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:30:06.380 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.826 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:30:06.407 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.827 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:30:06.432 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:45:01.853 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:30:06.457 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.583 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:38:28.164 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.593 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:38:28.173 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.618 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:38:28.223 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.641 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:38:28.246 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.649 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:38:28.254 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.661 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:38:28.266 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.814 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:38:28.395 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.895 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:38:28.500 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:23.896 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:38:28.502 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.801 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:38:47.382 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.809 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:38:47.391 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.818 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:38:47.423 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.821 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:38:47.426 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.824 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:38:47.429 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.828 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:38:47.433 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.841 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:38:47.422 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.858 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:38:47.463 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:53:42.867 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:38:47.472 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.073 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:43:59.655 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.086 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:43:59.668 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.088 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:43:59.693 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.093 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:43:59.698 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.096 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:43:59.701 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.104 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:43:59.710 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.319 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:43:59.901 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.416 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:44:00.021 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T12:58:55.430 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:44:00.035 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.668 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:46:50.250 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.717 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:46:50.287 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.732 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:46:50.337 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.745 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:46:50.351 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.753 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:46:50.358 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.754 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:46:50.360 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.798 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:46:50.380 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.846 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:46:50.452 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:01:45.856 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:46:50.462 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.405 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:49:07.987 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.422 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:49:08.004 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.444 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:49:08.049 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.448 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:49:08.053 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.452 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:49:08.058 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.460 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:49:08.066 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.548 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:49:08.152 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.581 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:49:08.186 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:04:03.583 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:49:08.166 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.586 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:54:21.168 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.596 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:54:21.201 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.603 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:54:21.209 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.607 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:54:21.190 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.608 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:54:21.213 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.614 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:54:21.220 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.799 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:54:21.404 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.813 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:54:21.418 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:09:16.845 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:54:21.412 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.637 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:55:12.219 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.653 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:55:12.235 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.695 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:55:12.299 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.705 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:55:12.287 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.718 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:55:12.323 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.722 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:55:12.327 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.727 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:55:12.332 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.768 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:55:12.373 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:10:07.770 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:55:12.375 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:27.856 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:59:32.438 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:27.874 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:59:32.456 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:27.878 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:59:32.484 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:27.882 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:59:32.487 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:27.888 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:59:32.493 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:27.888 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:59:32.494 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:28.059 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:03:59:32.664 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:28.124 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:03:59:32.730 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:14:28.185 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:03:59:32.767 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.500 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:03:34.082 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.507 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:03:34.089 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.544 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:03:34.150 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.556 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:03:34.162 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.573 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:03:34.179 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.587 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:03:34.193 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.677 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:03:34.260 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.691 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:03:34.296 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:18:29.740 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:03:34.346 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.838 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:04:42.421 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.847 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:04:42.429 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.853 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:04:42.458 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.857 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:04:42.463 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.858 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:04:42.464 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.870 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:04:42.475 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.913 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:04:42.495 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.917 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:04:42.521 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:19:37.925 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:04:42.530 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.013 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:09:54.595 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.022 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:09:54.605 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.032 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:09:54.638 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.033 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:09:54.639 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.037 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:09:54.643 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.040 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:09:54.646 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.245 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:09:54.851 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.246 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:09:54.828 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:24:50.264 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:09:54.870 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.339 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:11:55.922 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.350 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:11:55.932 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.388 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:11:55.994 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.402 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:11:56.008 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.404 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:11:56.010 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.422 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:11:56.028 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.472 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:11:56.054 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.516 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:11:56.122 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:26:51.521 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:11:56.126 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.109 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:15:06.691 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.121 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:15:06.727 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.122 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:15:06.705 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.123 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:15:06.729 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.128 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:15:06.734 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.132 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:15:06.738 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.244 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:15:06.826 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.286 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:15:06.892 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:30:02.291 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:15:06.897 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.119 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:20:17.701 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.175 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:20:17.757 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.212 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:20:17.817 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.218 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:20:17.824 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.225 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:20:17.831 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.256 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:20:17.862 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.306 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:20:17.888 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.382 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:20:17.988 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:13.383 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:20:17.989 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:14.989 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:20:19.572 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:15.000 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:20:19.582 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:15.065 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:20:19.647 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:15.147 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:20:19.753 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:15.149 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:20:19.754 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:15.162 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:20:19.768 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:15.208 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:20:19.813 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:15.209 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:20:19.815 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:35:15.222 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:20:19.828 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.293 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:24:33.875 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.301 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:24:33.883 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.384 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:24:33.990 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.386 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:24:33.992 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.387 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:24:33.993 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.389 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:24:33.995 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.501 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:24:34.083 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.538 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:24:34.144 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:39:29.543 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:24:34.149 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.875 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:25:30.457 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.892 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:25:30.474 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.896 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:25:30.502 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.897 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:25:30.503 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.898 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:25:30.504 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.902 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:25:30.508 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.973 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:25:30.579 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.975 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:25:30.581 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:40:25.981 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:25:30.563 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.263 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:28:39.845 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.307 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:28:39.888 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.310 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:28:39.916 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.312 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:28:39.918 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.328 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:28:39.934 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.383 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:28:39.989 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.399 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:28:40.005 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.443 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:28:40.025 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:43:35.500 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:28:40.106 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:36.914 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:30:41.496 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:36.925 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:30:41.507 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:36.934 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:30:41.540 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:36.942 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:30:41.548 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:36.945 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:30:41.551 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:36.957 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:30:41.562 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:37.056 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:30:41.638 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:37.059 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:30:41.665 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:45:37.085 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:30:41.691 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.175 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:35:53.757 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.178 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:35:53.760 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.192 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:35:53.797 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.193 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:35:53.799 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.196 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:35:53.802 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.203 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:35:53.809 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.370 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:35:53.976 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.392 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:35:53.995 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:50:49.451 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:35:54.033 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.872 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:37:01.454 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.883 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:37:01.466 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.904 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:37:01.510 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.909 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:37:01.514 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.913 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:37:01.519 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.934 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:37:01.539 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.942 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:37:01.524 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.967 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:37:01.573 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:51:56.977 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:37:01.583 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.508 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:41:06.090 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.519 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:41:06.102 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.536 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:41:06.143 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.544 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:41:06.151 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.545 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:41:06.151 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.552 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:41:06.158 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.676 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:41:06.258 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.714 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:41:06.320 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T13:56:01.749 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:41:06.356 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.495 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:45:23.078 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.532 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:45:23.115 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.609 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:45:23.216 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.611 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:45:23.218 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.613 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:45:23.220 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.621 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:45:23.228 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.693 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:45:23.276 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.726 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:45:23.332 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:00:18.732 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:45:23.339 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.883 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:46:14.466 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.889 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:46:14.472 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.906 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:46:14.513 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.910 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:46:14.517 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.940 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:46:14.523 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.941 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:46:14.548 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.947 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:46:14.554 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.976 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:46:14.583 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:01:09.992 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:46:14.599 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.053 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:51:28.636 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.064 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:51:28.647 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.068 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:51:28.675 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.074 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:51:28.682 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.075 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:51:28.682 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.081 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:51:28.689 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.270 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:51:28.877 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.291 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:51:28.874 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:06:24.316 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:51:28.923 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.447 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:53:45.029 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.502 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:53:45.085 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.533 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:53:45.140 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.535 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:53:45.142 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.539 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:53:45.147 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.552 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:53:45.160 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.599 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:53:45.206 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.622 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:53:45.205 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:08:40.630 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:53:45.238 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.031 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:56:40.613 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.040 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:56:40.624 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.048 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:56:40.656 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.055 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:56:40.663 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.057 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:56:40.664 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.064 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:56:40.671 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.197 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:04:56:40.780 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.279 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:04:56:40.887 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:11:36.288 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:04:56:40.895 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:22.148 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_load_graph ... ok 2019-11-25T14:16:22.158 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_leaf_node ... ok 2019-11-25T14:16:22.158 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_node ... ok 2019-11-25T14:16:22.158 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_bad_node ... ok 2019-11-25T14:16:22.158 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_descend_nonexistant_child ... ok 2019-11-25T14:16:22.158 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable ... ok 2019-11-25T14:16:22.159 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary ... ok 2019-11-25T14:16:22.528 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_printable_no_whitespace ... ok 2019-11-25T14:16:22.696 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_binary_no_whitespace ... ok 2019-11-25T14:16:23.084 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_args ... ok 2019-11-25T14:16:23.128 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_no_charset ... ok 2019-11-25T14:16:23.162 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_exact_length ... ok 2019-11-25T14:16:23.162 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_bad_charset ... ok 2019-11-25T14:16:23.163 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_random_missing_length ... ok 2019-11-25T14:16:23.163 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_assemble_decision ... ok 2019-11-25T14:16:23.163 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_escape ... ok 2019-11-25T14:16:23.163 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect ... ok 2019-11-25T14:16:23.163 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_indirect_double ... ok 2019-11-25T14:16:23.163 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive ... ok 2019-11-25T14:16:23.163 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_mutual ... ok 2019-11-25T14:16:23.164 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_recursive_not_too_eager ... ok 2019-11-25T14:16:23.164 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_make_choice_unweighted_with_space ... ok 2019-11-25T14:16:23.173 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_choices ... ok 2019-11-25T14:16:23.174 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_choices ... ok 2019-11-25T14:16:23.174 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_null_choices ... ok 2019-11-25T14:16:23.174 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_null_child ... ok 2019-11-25T14:16:23.182 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_weighted_set ... ok 2019-11-25T14:16:23.182 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_header_presence ... ok 2019-11-25T14:16:23.183 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_duplicate_header ... ok 2019-11-25T14:16:23.183 INFO:teuthology.orchestra.run.smithi153.stderr:s3tests_boto3.fuzz.test.test_fuzzer.test_expand_headers ... ok 2019-11-25T14:16:23.215 INFO:teuthology.orchestra.run.smithi153.stderr:test_realistic.TestFileValidator.test_new_file_is_valid ... ok 2019-11-25T14:16:23.250 INFO:teuthology.orchestra.run.smithi153.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_on_several_calls ... ok 2019-11-25T14:16:23.250 INFO:teuthology.orchestra.run.smithi153.stderr:test_realistic.TestFileValidator.test_new_file_is_valid_when_size_is_1 ... ok 2019-11-25T14:16:23.518 INFO:teuthology.orchestra.run.smithi153.stderr:test_realistic.TestFiles.test_random_file_valid ... ok 2019-11-25T14:16:23.518 INFO:teuthology.orchestra.run.smithi153.stderr: 2019-11-25T14:16:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:---------------------------------------------------------------------- 2019-11-25T14:16:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:Ran 611 tests in 13792.196s 2019-11-25T14:16:23.519 INFO:teuthology.orchestra.run.smithi153.stderr: 2019-11-25T14:16:23.519 INFO:teuthology.orchestra.run.smithi153.stderr:OK (SKIP=75) 2019-11-25T14:16:23.571 DEBUG:tasks.s3tests:Scanning radosgw logs for leaked encryption keys... 2019-11-25T14:16:23.571 INFO:teuthology.orchestra.run.smithi153:> grep --binary-files=text pO3upElrwuEXSoFwCfnZPdSsmt/xWeFa0N9KgDijwVs= /var/log/ceph/rgw.ceph.client.0.log 2019-11-25T14:16:23.695 DEBUG:teuthology.orchestra.run:got remote process result: 1 2019-11-25T14:16:23.695 INFO:tasks.s3tests:Cleaning up boto... 2019-11-25T14:16:23.695 INFO:teuthology.orchestra.run.smithi153:> rm /home/ubuntu/cephtest/boto.cfg 2019-11-25T14:16:23.756 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid bar.client.0 --purge-data --cluster ceph 2019-11-25T14:16:23.881 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:23.879+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:23.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:23.918+0000 7f4893078ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:23.920 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:23.918+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:23.931 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:23.928+0000 7f4893078ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:23.931 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:23.928+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:23.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:23.938+0000 7f4893078ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:23.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:23.938+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:23.966 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:23.964+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:24.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.185+0000 7f4893078ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T14:16:24.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.185+0000 7f4893078ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T14:16:24.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.185+0000 7f4893078ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T14:16:24.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.185+0000 7f4893078ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T14:16:24.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.185+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.208 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.203+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:24.208 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.203+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.224+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:24.226 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.224+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.240 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.237+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:24.240 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.237+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.257 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.255+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:24.258 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.255+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.268 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.266+0000 7f4893078ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:24.269 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.266+0000 7f4893078ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T14:16:24.269 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.266+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.274+0000 7f4893078ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:24.277 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.274+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.288 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.286+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:24.289 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.286+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.297+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:24.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.297+0000 7f4893078ec0 20 zonegroup default 2019-11-25T14:16:24.300 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.297+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.305+0000 7f4893078ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:24.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.305+0000 7f4893078ec0 10 Cannot find current period zone using local zone 2019-11-25T14:16:24.307 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.305+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.311+0000 7f4893078ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:24.319 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.311+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.321+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:24.324 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.321+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:24.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.336+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:24.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.336+0000 7f4893078ec0 20 zone default found 2019-11-25T14:16:24.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.336+0000 7f4893078ec0 20 started sync module instance, tier type = 2019-11-25T14:16:24.340 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.336+0000 7f4893078ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T14:16:24.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.584+0000 7f48802a6700 0 -- 172.21.15.153:0/1691459408 >> [v2:172.21.15.81:6808/9717,v1:172.21.15.81:6809/9717] conn(0x56063ffb6350 msgr2=0x56063ffbed70 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T14:16:24.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.699+0000 7f4893078ec0 20 add_watcher() i=0 2019-11-25T14:16:24.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.699+0000 7f4893078ec0 20 add_watcher() i=1 2019-11-25T14:16:24.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.699+0000 7f4893078ec0 20 add_watcher() i=2 2019-11-25T14:16:24.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.699+0000 7f4893078ec0 20 add_watcher() i=3 2019-11-25T14:16:24.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.699+0000 7f4893078ec0 20 add_watcher() i=4 2019-11-25T14:16:24.701 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.699+0000 7f4893078ec0 20 add_watcher() i=5 2019-11-25T14:16:24.702 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.699+0000 7f4893078ec0 20 add_watcher() i=6 2019-11-25T14:16:24.741 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.739+0000 7f4893078ec0 20 add_watcher() i=7 2019-11-25T14:16:24.741 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.739+0000 7f4893078ec0 2 all 8 watchers are set, enabling cache 2019-11-25T14:16:24.741 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.739+0000 7f4850fe1700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T14:16:24.756 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:24.755+0000 7f487faa5700 0 -- 172.21.15.153:0/55396144 >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x56063ffde010 msgr2=0x56063ffe0520 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T14:16:26.826 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.825+0000 7f47c2ffd700 20 reqs_thread_entry: start 2019-11-25T14:16:26.827 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.825+0000 7f47c0ff9700 20 reqs_thread_entry: start 2019-11-25T14:16:26.827 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.825+0000 7f4893078ec0 20 init_complete bucket index max shards: 0 2019-11-25T14:16:26.827 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.826+0000 7f47b6ffd700 20 reqs_thread_entry: start 2019-11-25T14:16:26.828 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.826+0000 7f4893078ec0 20 get_system_obj_state: rctx=0x7ffd15fb8498 obj=default.rgw.meta:users.uid:bar.client.0 state=0x5606400528f0 s->prefetch_data=0 2019-11-25T14:16:26.828 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.826+0000 7f4893078ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-11-25T14:16:26.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.848+0000 7f4893078ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x16 2019-11-25T14:16:26.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.848+0000 7f4893078ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T14:16:26.852 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.848+0000 7f4893078ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T14:16:26.853 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.848+0000 7f4893078ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T14:16:26.853 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.848+0000 7f4893078ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-25T14:16:26.853 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.848+0000 7f4893078ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:26.871 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.869+0000 7f4893078ec0 20 rados_obj.operate() r=0 bl.length=446 2019-11-25T14:16:26.871 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.869+0000 7f4893078ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x11 2019-11-25T14:16:26.871 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:26.869+0000 7f4893078ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T14:16:27.030 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:27.028+0000 7f4893078ec0 10 removing key index: JBHEYAKBJIAFTAJDWIHI 2019-11-25T14:16:27.030 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:27.028+0000 7f4893078ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=170 2019-11-25T14:16:27.041 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:27.039+0000 7f4865ffb700 10 RGWWatcher::handle_notify() notify_id 592705487014 cookie 94584843285008 notifier 4679 bl.length()=170 2019-11-25T14:16:27.458 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:27.457+0000 7f4893078ec0 10 removing email index: bar.client.0+test@test.test 2019-11-25T14:16:27.459 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:27.457+0000 7f4893078ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=178 2019-11-25T14:16:27.506 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:27.505+0000 7f4865ffb700 10 RGWWatcher::handle_notify() notify_id 592705487418 cookie 94584843288368 notifier 4679 bl.length()=178 2019-11-25T14:16:28.008 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:27.999+0000 7f4893078ec0 10 removing user buckets index 2019-11-25T14:16:28.008 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.000+0000 7f4893078ec0 10 distributing notification oid=default.rgw.control:notify.2 bl.length()=169 2019-11-25T14:16:28.023 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.022+0000 7f4865ffb700 10 RGWWatcher::handle_notify() notify_id 592705487281 cookie 94584843289568 notifier 4679 bl.length()=169 2019-11-25T14:16:28.470 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.469+0000 7f4893078ec0 10 removing user index: bar.client.0 2019-11-25T14:16:28.471 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.469+0000 7f4893078ec0 10 removing default.rgw.meta+users.uid+bar.client.0 from cache 2019-11-25T14:16:28.471 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.469+0000 7f4893078ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=161 2019-11-25T14:16:28.489 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.488+0000 7f4865ffb700 10 RGWWatcher::handle_notify() notify_id 592705487419 cookie 94584843288368 notifier 4679 bl.length()=161 2019-11-25T14:16:28.764 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.763+0000 7f4893078ec0 20 remove_watcher() i=0 2019-11-25T14:16:28.764 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.763+0000 7f4893078ec0 2 removed watcher, disabling cache 2019-11-25T14:16:28.804 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.803+0000 7f4893078ec0 20 remove_watcher() i=1 2019-11-25T14:16:28.849 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.848+0000 7f4893078ec0 20 remove_watcher() i=2 2019-11-25T14:16:28.924 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.923+0000 7f4893078ec0 20 remove_watcher() i=3 2019-11-25T14:16:28.997 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:28.996+0000 7f4893078ec0 20 remove_watcher() i=4 2019-11-25T14:16:29.048 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.046+0000 7f4893078ec0 20 remove_watcher() i=5 2019-11-25T14:16:29.081 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.079+0000 7f4893078ec0 20 remove_watcher() i=6 2019-11-25T14:16:29.108 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.106+0000 7f4893078ec0 20 remove_watcher() i=7 2019-11-25T14:16:29.114 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid foo.client.0 --purge-data --cluster ceph 2019-11-25T14:16:29.225 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.223+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.251 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.250+0000 7f81a3a92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:29.252 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.250+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.259 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.258+0000 7f81a3a92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:29.260 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.258+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.268 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.267+0000 7f81a3a92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:29.269 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.267+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.291+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:29.498 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.497+0000 7f81a3a92ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T14:16:29.498 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.497+0000 7f81a3a92ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T14:16:29.498 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.497+0000 7f81a3a92ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T14:16:29.499 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.497+0000 7f81a3a92ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T14:16:29.499 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.497+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.510 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.509+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:29.510 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.509+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.531 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.530+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:29.531 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.530+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.549 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.548+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:29.549 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.548+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.571 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.568+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:29.572 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.568+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.583 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.581+0000 7f81a3a92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:29.583 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.581+0000 7f81a3a92ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T14:16:29.583 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.581+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.605+0000 7f81a3a92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:29.609 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.605+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.615+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:29.616 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.615+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.627+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.627+0000 7f81a3a92ec0 20 zonegroup default 2019-11-25T14:16:29.629 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.627+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.635+0000 7f81a3a92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.635+0000 7f81a3a92ec0 10 Cannot find current period zone using local zone 2019-11-25T14:16:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.635+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.656 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.641+0000 7f81a3a92ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.641+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.654+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:29.657 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.654+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:29.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.663+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:29.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.663+0000 7f81a3a92ec0 20 zone default found 2019-11-25T14:16:29.665 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.663+0000 7f81a3a92ec0 20 started sync module instance, tier type = 2019-11-25T14:16:29.666 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:29.663+0000 7f81a3a92ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T14:16:30.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.163+0000 7f81a3a92ec0 20 add_watcher() i=0 2019-11-25T14:16:30.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.163+0000 7f81a3a92ec0 20 add_watcher() i=1 2019-11-25T14:16:30.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.163+0000 7f81a3a92ec0 20 add_watcher() i=2 2019-11-25T14:16:30.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.163+0000 7f81a3a92ec0 20 add_watcher() i=3 2019-11-25T14:16:30.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.163+0000 7f81a3a92ec0 20 add_watcher() i=4 2019-11-25T14:16:30.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.163+0000 7f81a3a92ec0 20 add_watcher() i=5 2019-11-25T14:16:30.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.163+0000 7f81a3a92ec0 20 add_watcher() i=6 2019-11-25T14:16:30.207 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.206+0000 7f81a3a92ec0 20 add_watcher() i=7 2019-11-25T14:16:30.207 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.206+0000 7f81a3a92ec0 2 all 8 watchers are set, enabling cache 2019-11-25T14:16:30.207 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.206+0000 7f81617e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T14:16:30.222 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.221+0000 7f8190cc0700 0 -- >> v1:172.21.15.153:6789/0 conn(0x55d388f449d0 legacy=0x55d388f46df0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T14:16:30.410 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:30.408+0000 7f81914c1700 0 -- 172.21.15.153:0/329276553 >> [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] conn(0x55d388f565e0 msgr2=0x55d388f58a00 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T14:16:32.411 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.410+0000 7f80d37fe700 20 reqs_thread_entry: start 2019-11-25T14:16:32.412 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.410+0000 7f80d17fa700 20 reqs_thread_entry: start 2019-11-25T14:16:32.412 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.410+0000 7f81a3a92ec0 20 init_complete bucket index max shards: 0 2019-11-25T14:16:32.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.411+0000 7f80bf7fe700 20 reqs_thread_entry: start 2019-11-25T14:16:32.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.411+0000 7f81a3a92ec0 20 get_system_obj_state: rctx=0x7ffd37c33858 obj=default.rgw.meta:users.uid:foo.client.0 state=0x55d388fbcd90 s->prefetch_data=0 2019-11-25T14:16:32.413 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.411+0000 7f81a3a92ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-11-25T14:16:32.435 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.433+0000 7f81a3a92ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x16 2019-11-25T14:16:32.435 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.433+0000 7f81a3a92ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T14:16:32.435 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.433+0000 7f81a3a92ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T14:16:32.436 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.433+0000 7f81a3a92ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T14:16:32.436 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.433+0000 7f81a3a92ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-25T14:16:32.436 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.433+0000 7f81a3a92ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:32.456 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.454+0000 7f81a3a92ec0 20 rados_obj.operate() r=0 bl.length=446 2019-11-25T14:16:32.456 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.454+0000 7f81a3a92ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x11 2019-11-25T14:16:32.456 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.454+0000 7f81a3a92ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T14:16:32.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.562+0000 7f81a3a92ec0 10 removing key index: LBFZNLAOSMYTHLEKZWHV 2019-11-25T14:16:32.563 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.562+0000 7f81a3a92ec0 10 distributing notification oid=default.rgw.control:notify.4 bl.length()=170 2019-11-25T14:16:32.573 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.572+0000 7f81867fc700 10 RGWWatcher::handle_notify() notify_id 592705487282 cookie 94367024074144 notifier 4688 bl.length()=170 2019-11-25T14:16:32.766 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.765+0000 7f81a3a92ec0 10 removing email index: foo.client.0+test@test.test 2019-11-25T14:16:32.767 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.765+0000 7f81a3a92ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=178 2019-11-25T14:16:32.775 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:32.774+0000 7f81867fc700 10 RGWWatcher::handle_notify() notify_id 592705486899 cookie 94367024080336 notifier 4688 bl.length()=178 2019-11-25T14:16:33.239 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:33.238+0000 7f81a3a92ec0 10 removing user buckets index 2019-11-25T14:16:33.240 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:33.238+0000 7f81a3a92ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=169 2019-11-25T14:16:33.265 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:33.261+0000 7f81867fc700 10 RGWWatcher::handle_notify() notify_id 592705487420 cookie 94367024068560 notifier 4688 bl.length()=169 2019-11-25T14:16:33.640 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:33.639+0000 7f81a3a92ec0 10 removing user index: foo.client.0 2019-11-25T14:16:33.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:33.639+0000 7f81a3a92ec0 10 removing default.rgw.meta+users.uid+foo.client.0 from cache 2019-11-25T14:16:33.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:33.639+0000 7f81a3a92ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=161 2019-11-25T14:16:33.651 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:33.649+0000 7f81867fc700 10 RGWWatcher::handle_notify() notify_id 592705487015 cookie 94367024065200 notifier 4688 bl.length()=161 2019-11-25T14:16:34.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.425+0000 7f81a3a92ec0 20 remove_watcher() i=0 2019-11-25T14:16:34.427 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.425+0000 7f81a3a92ec0 2 removed watcher, disabling cache 2019-11-25T14:16:34.484 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.483+0000 7f81a3a92ec0 20 remove_watcher() i=1 2019-11-25T14:16:34.528 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.527+0000 7f81a3a92ec0 20 remove_watcher() i=2 2019-11-25T14:16:34.588 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.586+0000 7f81a3a92ec0 20 remove_watcher() i=3 2019-11-25T14:16:34.631 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.630+0000 7f81a3a92ec0 20 remove_watcher() i=4 2019-11-25T14:16:34.688 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.687+0000 7f81a3a92ec0 20 remove_watcher() i=5 2019-11-25T14:16:34.730 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.729+0000 7f81a3a92ec0 20 remove_watcher() i=6 2019-11-25T14:16:34.767 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.765+0000 7f81a3a92ec0 20 remove_watcher() i=7 2019-11-25T14:16:34.776 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 user rm --uid 'testx$tenanteduser.client.0' --purge-data --cluster ceph 2019-11-25T14:16:34.902 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.900+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:34.935 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.934+0000 7f95ab687ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:34.935 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.934+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:34.943 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.942+0000 7f95ab687ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:34.944 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.942+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:34.953 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.952+0000 7f95ab687ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:34.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.952+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:34.985 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:34.984+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.174+0000 7f95ab687ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.174+0000 7f95ab687ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.174+0000 7f95ab687ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.174+0000 7f95ab687ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T14:16:35.176 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.174+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.187+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:35.188 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.187+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.202 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.201+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:35.203 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.201+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.219 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.218+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:35.219 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.218+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.231 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.229+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:35.231 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.229+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.240 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.239+0000 7f95ab687ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:35.240 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.239+0000 7f95ab687ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T14:16:35.240 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.239+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.248 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.246+0000 7f95ab687ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:35.248 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.246+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.265 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.264+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:35.265 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.264+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.279+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:35.280 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.279+0000 7f95ab687ec0 20 zonegroup default 2019-11-25T14:16:35.281 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.279+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.286+0000 7f95ab687ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:35.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.286+0000 7f95ab687ec0 10 Cannot find current period zone using local zone 2019-11-25T14:16:35.287 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.286+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.293 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.292+0000 7f95ab687ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:35.294 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.292+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.304 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.303+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:35.305 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.303+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:35.320 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.319+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:35.320 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.319+0000 7f95ab687ec0 20 zone default found 2019-11-25T14:16:35.321 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.319+0000 7f95ab687ec0 20 started sync module instance, tier type = 2019-11-25T14:16:35.321 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.319+0000 7f95ab687ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T14:16:35.641 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.640+0000 7f95988b5700 0 -- 172.21.15.153:0/2164887177 >> [v2:172.21.15.153:6802/9694,v1:172.21.15.153:6803/9694] conn(0x7f9588007080 msgr2=0x7f9588007490 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T14:16:35.682 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.681+0000 7f95ab687ec0 20 add_watcher() i=0 2019-11-25T14:16:35.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.681+0000 7f95ab687ec0 20 add_watcher() i=1 2019-11-25T14:16:35.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.681+0000 7f95ab687ec0 20 add_watcher() i=2 2019-11-25T14:16:35.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.681+0000 7f95ab687ec0 20 add_watcher() i=3 2019-11-25T14:16:35.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.681+0000 7f95ab687ec0 20 add_watcher() i=4 2019-11-25T14:16:35.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.681+0000 7f95ab687ec0 20 add_watcher() i=5 2019-11-25T14:16:35.684 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.682+0000 7f95ab687ec0 20 add_watcher() i=6 2019-11-25T14:16:35.711 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.710+0000 7f95ab687ec0 20 add_watcher() i=7 2019-11-25T14:16:35.711 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.710+0000 7f95ab687ec0 2 all 8 watchers are set, enabling cache 2019-11-25T14:16:35.712 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.710+0000 7f95697e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T14:16:35.728 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:35.726+0000 7f95980b4700 0 -- 172.21.15.153:0/3602000374 >> v2:172.21.15.81:3300/0 conn(0x55c4e70a3550 msgr2=0x55c4e70a5a60 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T14:16:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.149+0000 7f94db7fe700 20 reqs_thread_entry: start 2019-11-25T14:16:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.149+0000 7f94d97fa700 20 reqs_thread_entry: start 2019-11-25T14:16:37.150 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.149+0000 7f95ab687ec0 20 init_complete bucket index max shards: 0 2019-11-25T14:16:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.149+0000 7f94c77fe700 20 reqs_thread_entry: start 2019-11-25T14:16:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.149+0000 7f95ab687ec0 20 get_system_obj_state: rctx=0x7ffed4c5f128 obj=default.rgw.meta:users.uid:testx$tenanteduser.client.0 state=0x55c4e7117e20 s->prefetch_data=0 2019-11-25T14:16:37.151 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.149+0000 7f95ab687ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : miss 2019-11-25T14:16:37.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.168+0000 7f95ab687ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x16 2019-11-25T14:16:37.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.168+0000 7f95ab687ec0 10 adding default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T14:16:37.169 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.168+0000 7f95ab687ec0 10 updating xattr: name=ceph.objclass.version bl.length()=42 2019-11-25T14:16:37.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.168+0000 7f95ab687ec0 20 get_system_obj_state: s->obj_tag was set empty 2019-11-25T14:16:37.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.168+0000 7f95ab687ec0 10 cache get: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 : type miss (requested=0x11, cached=0x16) 2019-11-25T14:16:37.170 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.168+0000 7f95ab687ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:37.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.185+0000 7f95ab687ec0 20 rados_obj.operate() r=0 bl.length=505 2019-11-25T14:16:37.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.185+0000 7f95ab687ec0 10 cache put: name=default.rgw.meta+users.uid+testx$tenanteduser.client.0 info.flags=0x11 2019-11-25T14:16:37.186 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.185+0000 7f95ab687ec0 10 moving default.rgw.meta+users.uid+testx$tenanteduser.client.0 to cache LRU end 2019-11-25T14:16:37.198 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.196+0000 7f95ab687ec0 10 removing key index: ZWMLDUAQBGMKDPFMGHKS 2019-11-25T14:16:37.198 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.196+0000 7f95ab687ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=170 2019-11-25T14:16:37.213 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.212+0000 7f958e7fc700 10 RGWWatcher::handle_notify() notify_id 592705487421 cookie 94304178040736 notifier 4700 bl.length()=170 2019-11-25T14:16:37.745 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.744+0000 7f95ab687ec0 10 removing email index: testx$tenanteduser.client.0+test@test.test 2019-11-25T14:16:37.746 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.744+0000 7f95ab687ec0 10 distributing notification oid=default.rgw.control:notify.7 bl.length()=193 2019-11-25T14:16:37.784 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:37.783+0000 7f958e7fc700 10 RGWWatcher::handle_notify() notify_id 592705487016 cookie 94304178049328 notifier 4700 bl.length()=193 2019-11-25T14:16:38.525 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:38.524+0000 7f95ab687ec0 10 removing user buckets index 2019-11-25T14:16:38.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:38.524+0000 7f95ab687ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=184 2019-11-25T14:16:38.544 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:38.542+0000 7f958e7fc700 10 RGWWatcher::handle_notify() notify_id 592705486900 cookie 94304178048352 notifier 4700 bl.length()=184 2019-11-25T14:16:38.812 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:38.811+0000 7f95ab687ec0 10 removing user index: testx$tenanteduser.client.0 2019-11-25T14:16:38.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:38.811+0000 7f95ab687ec0 10 removing default.rgw.meta+users.uid+testx$tenanteduser.client.0 from cache 2019-11-25T14:16:38.813 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:38.811+0000 7f95ab687ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=176 2019-11-25T14:16:38.833 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:38.832+0000 7f958e7fc700 10 RGWWatcher::handle_notify() notify_id 592705487422 cookie 94304178040736 notifier 4700 bl.length()=176 2019-11-25T14:16:40.071 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.069+0000 7f95ab687ec0 20 remove_watcher() i=0 2019-11-25T14:16:40.071 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.069+0000 7f95ab687ec0 2 removed watcher, disabling cache 2019-11-25T14:16:40.117 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.107+0000 7f95ab687ec0 20 remove_watcher() i=1 2019-11-25T14:16:40.155 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.154+0000 7f95ab687ec0 20 remove_watcher() i=2 2019-11-25T14:16:40.208 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.202+0000 7f95ab687ec0 20 remove_watcher() i=3 2019-11-25T14:16:40.246 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.243+0000 7f95ab687ec0 20 remove_watcher() i=4 2019-11-25T14:16:40.283 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.282+0000 7f95ab687ec0 20 remove_watcher() i=5 2019-11-25T14:16:40.424 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.422+0000 7f95ab687ec0 20 remove_watcher() i=6 2019-11-25T14:16:40.467 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:40.466+0000 7f95ab687ec0 20 remove_watcher() i=7 2019-11-25T14:16:40.475 INFO:tasks.s3tests:Removing s3-tests... 2019-11-25T14:16:40.476 INFO:teuthology.orchestra.run.smithi153:> rm -rf /home/ubuntu/cephtest/s3-tests 2019-11-25T14:16:40.614 INFO:teuthology.run_tasks:Running task swift... 2019-11-25T14:16:40.644 INFO:tasks.swift:clients=['client.0'] 2019-11-25T14:16:40.644 INFO:tasks.swift:Downloading swift... 2019-11-25T14:16:40.645 INFO:teuthology.orchestra.run.smithi153:> git clone -b ceph-master git://git.ceph.com/git/swift.git /home/ubuntu/cephtest/swift 2019-11-25T14:16:40.755 INFO:teuthology.orchestra.run.smithi153.stdout:Cloning into '/home/ubuntu/cephtest/swift'... 2019-11-25T14:16:41.245 INFO:tasks.swift:Creating rgw users... 2019-11-25T14:16:41.245 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser foo.client.0:foo --display-name 'Mr. foo.client.0 foo' --secret 54Zm0aFiHGvRBJHKyGqShl2mkv0i/6XfENwcQtb82Isq7xJojRhZTg== --email foo.client.0+test@test.test --key-type swift --access full 2019-11-25T14:16:41.433 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:41.432+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:41.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:41.479+0000 7f17124f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:41.480 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:41.479+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:41.490+0000 7f17124f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:41.492 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:41.490+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:41.504 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:41.502+0000 7f17124f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:41.504 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:41.502+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:41.532 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:41.531+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:42.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.003+0000 7f17124f1ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T14:16:42.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.003+0000 7f17124f1ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T14:16:42.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.003+0000 7f17124f1ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T14:16:42.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.003+0000 7f17124f1ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T14:16:42.005 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.003+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.018 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.017+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:42.019 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.017+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.029 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.028+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:42.029 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.028+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.039 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.037+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:42.039 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.038+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.050 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.049+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:42.050 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.049+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.059+0000 7f17124f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:42.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.059+0000 7f17124f1ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T14:16:42.060 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.059+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.068 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.066+0000 7f17124f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:42.068 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.066+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.077 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.076+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:42.078 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.076+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.091 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.090+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:42.091 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.090+0000 7f17124f1ec0 20 zonegroup default 2019-11-25T14:16:42.091 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.090+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.098 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.097+0000 7f17124f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:42.099 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.097+0000 7f17124f1ec0 10 Cannot find current period zone using local zone 2019-11-25T14:16:42.099 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.097+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.111 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.110+0000 7f17124f1ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:42.111 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.110+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.124 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.122+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:42.124 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.123+0000 7f17124f1ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:42.132 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.131+0000 7f17124f1ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:42.133 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.131+0000 7f17124f1ec0 20 zone default found 2019-11-25T14:16:42.133 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.131+0000 7f17124f1ec0 20 started sync module instance, tier type = 2019-11-25T14:16:42.133 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.131+0000 7f17124f1ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T14:16:42.525 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.523+0000 7f17124f1ec0 20 add_watcher() i=0 2019-11-25T14:16:42.525 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.523+0000 7f17124f1ec0 20 add_watcher() i=1 2019-11-25T14:16:42.525 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.523+0000 7f17124f1ec0 20 add_watcher() i=2 2019-11-25T14:16:42.525 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.523+0000 7f17124f1ec0 20 add_watcher() i=3 2019-11-25T14:16:42.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.523+0000 7f17124f1ec0 20 add_watcher() i=4 2019-11-25T14:16:42.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.523+0000 7f17124f1ec0 20 add_watcher() i=5 2019-11-25T14:16:42.526 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.523+0000 7f17124f1ec0 20 add_watcher() i=6 2019-11-25T14:16:42.555 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.554+0000 7f17124f1ec0 20 add_watcher() i=7 2019-11-25T14:16:42.556 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.554+0000 7f17124f1ec0 2 all 8 watchers are set, enabling cache 2019-11-25T14:16:42.556 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.554+0000 7f16cffdf700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T14:16:42.557 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.556+0000 7f16fef1e700 0 -- >> [v2:172.21.15.153:3301/0,v1:172.21.15.153:6790/0] conn(0x5567acf65e70 msgr2=0x5567acf68380 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=0).read_until injecting socket failure 2019-11-25T14:16:42.589 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:42.588+0000 7f16ff71f700 0 -- 172.21.15.153:0/1212101264 >> [v2:172.21.15.153:6800/8740,v1:172.21.15.153:6801/8740] conn(0x7f1640017570 msgr2=0x7f1640019a60 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T14:16:44.026 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.025+0000 7f163e7fc700 20 reqs_thread_entry: start 2019-11-25T14:16:44.026 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.025+0000 7f1637fff700 20 reqs_thread_entry: start 2019-11-25T14:16:44.027 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.025+0000 7f17124f1ec0 20 init_complete bucket index max shards: 0 2019-11-25T14:16:44.027 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.026+0000 7f16367fc700 20 reqs_thread_entry: start 2019-11-25T14:16:44.027 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.026+0000 7f17124f1ec0 20 get_system_obj_state: rctx=0x7ffc105de688 obj=default.rgw.meta:users.uid:foo.client.0 state=0x5567acfe0910 s->prefetch_data=0 2019-11-25T14:16:44.028 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.026+0000 7f17124f1ec0 10 cache get: name=default.rgw.meta+users.uid+foo.client.0 : miss 2019-11-25T14:16:44.036 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.035+0000 7f17124f1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x0 2019-11-25T14:16:44.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.035+0000 7f17124f1ec0 10 adding default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T14:16:44.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.035+0000 7f17124f1ec0 20 get_system_obj_state: rctx=0x7ffc105de698 obj=default.rgw.meta:users.email:foo.client.0+test@test.test state=0x5567acfe26a0 s->prefetch_data=0 2019-11-25T14:16:44.037 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.035+0000 7f17124f1ec0 10 cache get: name=default.rgw.meta+users.email+foo.client.0+test@test.test : miss 2019-11-25T14:16:44.045 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.043+0000 7f17124f1ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x0 2019-11-25T14:16:44.045 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.043+0000 7f17124f1ec0 10 adding default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-25T14:16:44.045 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.044+0000 7f17124f1ec0 20 get_system_obj_state: rctx=0x7ffc105de2a8 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x5567acfe26a0 s->prefetch_data=0 2019-11-25T14:16:44.045 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.044+0000 7f17124f1ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : miss 2019-11-25T14:16:44.053 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.052+0000 7f17124f1ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-11-25T14:16:44.053 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.052+0000 7f17124f1ec0 10 adding default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-11-25T14:16:44.053 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.052+0000 7f17124f1ec0 20 get_system_obj_state: rctx=0x7ffc105de428 obj=default.rgw.meta:users.swift:foo.client.0:foo state=0x5567acfe2950 s->prefetch_data=0 2019-11-25T14:16:44.053 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.052+0000 7f17124f1ec0 10 cache get: name=default.rgw.meta+users.swift+foo.client.0:foo : type miss (requested=0x6, cached=0x0) 2019-11-25T14:16:44.064 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.063+0000 7f17124f1ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x0 2019-11-25T14:16:44.064 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.063+0000 7f17124f1ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-11-25T14:16:44.110 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.109+0000 7f17124f1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-25T14:16:44.111 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.109+0000 7f17124f1ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T14:16:44.111 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.109+0000 7f17124f1ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=612 2019-11-25T14:16:44.119 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.118+0000 7f16e4ff9700 10 RGWWatcher::handle_notify() notify_id 592705487017 cookie 93903771723008 notifier 4706 bl.length()=612 2019-11-25T14:16:44.119 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.118+0000 7f16e4ff9700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-25T14:16:44.120 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.118+0000 7f16e4ff9700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T14:16:44.440 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.439+0000 7f17124f1ec0 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-11-25T14:16:44.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.439+0000 7f17124f1ec0 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-25T14:16:44.441 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.439+0000 7f17124f1ec0 10 distributing notification oid=default.rgw.control:notify.6 bl.length()=194 2019-11-25T14:16:44.451 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.450+0000 7f16e4ff9700 10 RGWWatcher::handle_notify() notify_id 592705486901 cookie 93903771736944 notifier 4706 bl.length()=194 2019-11-25T14:16:44.451 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.450+0000 7f16e4ff9700 10 cache put: name=default.rgw.meta+users.email+foo.client.0+test@test.test info.flags=0x7 2019-11-25T14:16:44.452 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.450+0000 7f16e4ff9700 10 moving default.rgw.meta+users.email+foo.client.0+test@test.test to cache LRU end 2019-11-25T14:16:44.668 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.665+0000 7f17124f1ec0 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-11-25T14:16:44.668 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.665+0000 7f17124f1ec0 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-11-25T14:16:44.668 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.665+0000 7f17124f1ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=183 2019-11-25T14:16:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.682+0000 7f16e4ff9700 10 RGWWatcher::handle_notify() notify_id 592705487423 cookie 93903771726336 notifier 4706 bl.length()=183 2019-11-25T14:16:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.682+0000 7f16e4ff9700 10 cache put: name=default.rgw.meta+users.swift+foo.client.0:foo info.flags=0x7 2019-11-25T14:16:44.683 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:44.682+0000 7f16e4ff9700 10 moving default.rgw.meta+users.swift+foo.client.0:foo to cache LRU end 2019-11-25T14:16:45.069 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.068+0000 7f17124f1ec0 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-25T14:16:45.069 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.068+0000 7f17124f1ec0 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T14:16:45.070 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.068+0000 7f17124f1ec0 10 distributing notification oid=default.rgw.control:notify.0 bl.length()=636 2019-11-25T14:16:45.091 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.090+0000 7f16e4ff9700 10 RGWWatcher::handle_notify() notify_id 592705487018 cookie 93903771723008 notifier 4706 bl.length()=636 2019-11-25T14:16:45.091 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.090+0000 7f16e4ff9700 10 cache put: name=default.rgw.meta+users.uid+foo.client.0 info.flags=0x17 2019-11-25T14:16:45.091 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.090+0000 7f16e4ff9700 10 moving default.rgw.meta+users.uid+foo.client.0 to cache LRU end 2019-11-25T14:16:45.643 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2019-11-25T14:16:45.643 INFO:teuthology.orchestra.run.smithi153.stdout: "user_id": "foo.client.0", 2019-11-25T14:16:45.644 INFO:teuthology.orchestra.run.smithi153.stdout: "display_name": "Mr. foo.client.0 foo", 2019-11-25T14:16:45.644 INFO:teuthology.orchestra.run.smithi153.stdout: "email": "foo.client.0+test@test.test", 2019-11-25T14:16:45.644 INFO:teuthology.orchestra.run.smithi153.stdout: "suspended": 0, 2019-11-25T14:16:45.644 INFO:teuthology.orchestra.run.smithi153.stdout: "max_buckets": 1000, 2019-11-25T14:16:45.644 INFO:teuthology.orchestra.run.smithi153.stdout: "subusers": [ 2019-11-25T14:16:45.644 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T14:16:45.645 INFO:teuthology.orchestra.run.smithi153.stdout: "id": "foo.client.0:foo", 2019-11-25T14:16:45.645 INFO:teuthology.orchestra.run.smithi153.stdout: "permissions": "full-control" 2019-11-25T14:16:45.645 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T14:16:45.645 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T14:16:45.645 INFO:teuthology.orchestra.run.smithi153.stdout: "keys": [], 2019-11-25T14:16:45.645 INFO:teuthology.orchestra.run.smithi153.stdout: "swift_keys": [ 2019-11-25T14:16:45.645 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T14:16:45.646 INFO:teuthology.orchestra.run.smithi153.stdout: "user": "foo.client.0:foo", 2019-11-25T14:16:45.646 INFO:teuthology.orchestra.run.smithi153.stdout: "secret_key": "54Zm0aFiHGvRBJHKyGqShl2mkv0i/6XfENwcQtb82Isq7xJojRhZTg==" 2019-11-25T14:16:45.646 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T14:16:45.646 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T14:16:45.646 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": [], 2019-11-25T14:16:45.646 INFO:teuthology.orchestra.run.smithi153.stdout: "op_mask": "read, write, delete", 2019-11-25T14:16:45.646 INFO:teuthology.orchestra.run.smithi153.stdout: "default_placement": "", 2019-11-25T14:16:45.647 INFO:teuthology.orchestra.run.smithi153.stdout: "default_storage_class": "", 2019-11-25T14:16:45.647 INFO:teuthology.orchestra.run.smithi153.stdout: "placement_tags": [], 2019-11-25T14:16:45.647 INFO:teuthology.orchestra.run.smithi153.stdout: "bucket_quota": { 2019-11-25T14:16:45.647 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T14:16:45.647 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T14:16:45.647 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T14:16:45.647 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T14:16:45.648 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T14:16:45.648 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T14:16:45.648 INFO:teuthology.orchestra.run.smithi153.stdout: "user_quota": { 2019-11-25T14:16:45.648 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T14:16:45.649 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T14:16:45.649 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T14:16:45.649 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T14:16:45.649 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T14:16:45.649 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T14:16:45.650 INFO:teuthology.orchestra.run.smithi153.stdout: "temp_url_keys": [], 2019-11-25T14:16:45.650 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "rgw", 2019-11-25T14:16:45.650 INFO:teuthology.orchestra.run.smithi153.stdout: "mfa_ids": [] 2019-11-25T14:16:45.650 INFO:teuthology.orchestra.run.smithi153.stdout:} 2019-11-25T14:16:45.650 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T14:16:45.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.702+0000 7f17124f1ec0 20 remove_watcher() i=0 2019-11-25T14:16:45.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.702+0000 7f17124f1ec0 2 removed watcher, disabling cache 2019-11-25T14:16:45.764 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.763+0000 7f17124f1ec0 20 remove_watcher() i=1 2019-11-25T14:16:45.818 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.816+0000 7f17124f1ec0 20 remove_watcher() i=2 2019-11-25T14:16:45.858 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.857+0000 7f17124f1ec0 20 remove_watcher() i=3 2019-11-25T14:16:45.896 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.894+0000 7f17124f1ec0 20 remove_watcher() i=4 2019-11-25T14:16:45.930 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.929+0000 7f17124f1ec0 20 remove_watcher() i=5 2019-11-25T14:16:45.980 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:45.978+0000 7f17124f1ec0 20 remove_watcher() i=6 2019-11-25T14:16:46.009 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.007+0000 7f17124f1ec0 20 remove_watcher() i=7 2019-11-25T14:16:46.017 INFO:teuthology.orchestra.run.smithi153:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage radosgw-admin -n client.0 --cluster ceph user create --subuser bar.client.0:bar --display-name 'Mr. bar.client.0 bar' --secret Oc+bjGv5QMe2VVR33uEMJY/0ElH/Y+VvxenR96n7sFgvLYpoWbJnjA== --email bar.client.0+test@test.test --key-type swift --access full 2019-11-25T14:16:46.143 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.141+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.173+0000 7ff2976a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:46.174 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.173+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.182 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.180+0000 7ff2976a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:46.182 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.180+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.189 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.188+0000 7ff2976a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:46.189 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.188+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.209 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.207+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:46.449 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.447+0000 7ff2976a2ec0 20 RGWRados::pool_iterate: got zonegroup_info.55cd03c4-4153-4163-b709-3fc06c8d160b 2019-11-25T14:16:46.449 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.447+0000 7ff2976a2ec0 20 RGWRados::pool_iterate: got zone_info.103c5a94-eb69-4977-8c90-9f6d634d8761 2019-11-25T14:16:46.449 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.447+0000 7ff2976a2ec0 20 RGWRados::pool_iterate: got zone_names.default 2019-11-25T14:16:46.449 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.447+0000 7ff2976a2ec0 20 RGWRados::pool_iterate: got zonegroups_names.default 2019-11-25T14:16:46.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.447+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.464+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:46.465 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.464+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.478 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.476+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:46.478 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.476+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.496 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.495+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:46.497 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.495+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.513+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:46.515 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.513+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.521+0000 7ff2976a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.521+0000 7ff2976a2ec0 10 cannot find current period zonegroup using local zonegroup 2019-11-25T14:16:46.523 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.521+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.533 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.532+0000 7ff2976a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:46.533 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.532+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.545 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.544+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:46.546 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.544+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.560 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.559+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=382 2019-11-25T14:16:46.561 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.559+0000 7ff2976a2ec0 20 zonegroup default 2019-11-25T14:16:46.561 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.559+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.570 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.569+0000 7ff2976a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:46.571 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.569+0000 7ff2976a2ec0 10 Cannot find current period zone using local zone 2019-11-25T14:16:46.571 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.569+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.577+0000 7ff2976a2ec0 20 rados_obj.operate() r=-2 bl.length=0 2019-11-25T14:16:46.585 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.577+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.593 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.592+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=46 2019-11-25T14:16:46.593 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.592+0000 7ff2976a2ec0 20 rados->read ofs=0 len=0 2019-11-25T14:16:46.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.602+0000 7ff2976a2ec0 20 rados_obj.operate() r=0 bl.length=951 2019-11-25T14:16:46.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.602+0000 7ff2976a2ec0 20 zone default found 2019-11-25T14:16:46.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.602+0000 7ff2976a2ec0 20 started sync module instance, tier type = 2019-11-25T14:16:46.603 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.602+0000 7ff2976a2ec0 20 started zone id=103c5a94-eb69-4977-8c90-9f6d634d8761 (name=default) with tier type = 2019-11-25T14:16:46.918 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:01:51.501 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:46.929 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:01:51.513 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:46.937 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:01:51.544 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:46.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.938+0000 7ff2976a2ec0 20 add_watcher() i=0 2019-11-25T14:16:46.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.938+0000 7ff2976a2ec0 20 add_watcher() i=1 2019-11-25T14:16:46.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.938+0000 7ff2976a2ec0 20 add_watcher() i=2 2019-11-25T14:16:46.940 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.938+0000 7ff2976a2ec0 20 add_watcher() i=3 2019-11-25T14:16:46.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.938+0000 7ff2976a2ec0 20 add_watcher() i=4 2019-11-25T14:16:46.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.938+0000 7ff2976a2ec0 20 add_watcher() i=5 2019-11-25T14:16:46.941 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.940+0000 7ff2976a2ec0 20 add_watcher() i=6 2019-11-25T14:16:46.942 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:01:51.550 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:46.942 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:01:51.550 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:46.945 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:01:51.553 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.974+0000 7ff2976a2ec0 20 add_watcher() i=7 2019-11-25T14:16:46.975 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.974+0000 7ff2976a2ec0 2 all 8 watchers are set, enabling cache 2019-11-25T14:16:46.976 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:46.974+0000 7ff2557e2700 2 RGWDataChangesLog::ChangesRenewThread: start 2019-11-25T14:16:47.124 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:01:51.727 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:47.145 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:01:51.752 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:47.289 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:01:51.872 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:16:47.289 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:47.288+0000 7ff2840cf700 0 -- 172.21.15.153:0/846095944 >> v1:172.21.15.153:6789/0 conn(0x563fb29e66b0 legacy=0x563fb29e8bc0 unknown :-1 s=STATE_CONNECTION_ESTABLISHED l=1)._try_send injecting socket failure 2019-11-25T14:16:47.504 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:47.503+0000 7ff2850d1700 0 -- 172.21.15.153:0/846095944 >> [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] conn(0x563fb29f5920 msgr2=0x563fb29f7d40 crc :-1 s=STATE_CONNECTION_ESTABLISHED l=1).read_until injecting socket failure 2019-11-25T14:16:48.636 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.634+0000 7ff1c77fe700 20 reqs_thread_entry: start 2019-11-25T14:16:48.636 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.635+0000 7ff1c57fa700 20 reqs_thread_entry: start 2019-11-25T14:16:48.636 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.635+0000 7ff2976a2ec0 20 init_complete bucket index max shards: 0 2019-11-25T14:16:48.637 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.635+0000 7ff1b37fe700 20 reqs_thread_entry: start 2019-11-25T14:16:48.637 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.635+0000 7ff2976a2ec0 20 get_system_obj_state: rctx=0x7ffdbd7a0308 obj=default.rgw.meta:users.uid:bar.client.0 state=0x563fb2a595b0 s->prefetch_data=0 2019-11-25T14:16:48.637 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.635+0000 7ff2976a2ec0 10 cache get: name=default.rgw.meta+users.uid+bar.client.0 : miss 2019-11-25T14:16:48.642 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.641+0000 7ff2976a2ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x0 2019-11-25T14:16:48.642 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.641+0000 7ff2976a2ec0 10 adding default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T14:16:48.643 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.641+0000 7ff2976a2ec0 20 get_system_obj_state: rctx=0x7ffdbd7a0318 obj=default.rgw.meta:users.email:bar.client.0+test@test.test state=0x563fb2a5b340 s->prefetch_data=0 2019-11-25T14:16:48.643 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.641+0000 7ff2976a2ec0 10 cache get: name=default.rgw.meta+users.email+bar.client.0+test@test.test : miss 2019-11-25T14:16:48.647 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.646+0000 7ff2976a2ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x0 2019-11-25T14:16:48.647 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.646+0000 7ff2976a2ec0 10 adding default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-25T14:16:48.647 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.646+0000 7ff2976a2ec0 20 get_system_obj_state: rctx=0x7ffdbd79ff28 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x563fb2a5b340 s->prefetch_data=0 2019-11-25T14:16:48.648 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.646+0000 7ff2976a2ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : miss 2019-11-25T14:16:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.653+0000 7ff2976a2ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-11-25T14:16:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.653+0000 7ff2976a2ec0 10 adding default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-11-25T14:16:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.653+0000 7ff2976a2ec0 20 get_system_obj_state: rctx=0x7ffdbd7a00a8 obj=default.rgw.meta:users.swift:bar.client.0:bar state=0x563fb2a5b5f0 s->prefetch_data=0 2019-11-25T14:16:48.654 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.653+0000 7ff2976a2ec0 10 cache get: name=default.rgw.meta+users.swift+bar.client.0:bar : type miss (requested=0x6, cached=0x0) 2019-11-25T14:16:48.658 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.657+0000 7ff2976a2ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x0 2019-11-25T14:16:48.658 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.657+0000 7ff2976a2ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-11-25T14:16:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.693+0000 7ff2976a2ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-25T14:16:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.693+0000 7ff2976a2ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T14:16:48.695 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.693+0000 7ff2976a2ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=612 2019-11-25T14:16:48.702 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.701+0000 7ff27a7fc700 10 RGWWatcher::handle_notify() notify_id 592705487424 cookie 94831579522624 notifier 5005 bl.length()=612 2019-11-25T14:16:48.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.701+0000 7ff27a7fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-25T14:16:48.703 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:48.701+0000 7ff27a7fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T14:16:49.007 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.006+0000 7ff2976a2ec0 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-11-25T14:16:49.008 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.006+0000 7ff2976a2ec0 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-25T14:16:49.008 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.006+0000 7ff2976a2ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=194 2019-11-25T14:16:49.023 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.022+0000 7ff27a7fc700 10 RGWWatcher::handle_notify() notify_id 592705487425 cookie 94831579522624 notifier 5005 bl.length()=194 2019-11-25T14:16:49.023 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.022+0000 7ff27a7fc700 10 cache put: name=default.rgw.meta+users.email+bar.client.0+test@test.test info.flags=0x7 2019-11-25T14:16:49.023 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.022+0000 7ff27a7fc700 10 moving default.rgw.meta+users.email+bar.client.0+test@test.test to cache LRU end 2019-11-25T14:16:49.712 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.711+0000 7ff2976a2ec0 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-11-25T14:16:49.712 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.711+0000 7ff2976a2ec0 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-11-25T14:16:49.713 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.711+0000 7ff2976a2ec0 10 distributing notification oid=default.rgw.control:notify.5 bl.length()=183 2019-11-25T14:16:49.732 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.731+0000 7ff27a7fc700 10 RGWWatcher::handle_notify() notify_id 592705487426 cookie 94831579532656 notifier 5005 bl.length()=183 2019-11-25T14:16:49.732 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.731+0000 7ff27a7fc700 10 cache put: name=default.rgw.meta+users.swift+bar.client.0:bar info.flags=0x7 2019-11-25T14:16:49.733 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.731+0000 7ff27a7fc700 10 moving default.rgw.meta+users.swift+bar.client.0:bar to cache LRU end 2019-11-25T14:16:49.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.952+0000 7ff2976a2ec0 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-25T14:16:49.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.952+0000 7ff2976a2ec0 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T14:16:49.954 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.952+0000 7ff2976a2ec0 10 distributing notification oid=default.rgw.control:notify.1 bl.length()=636 2019-11-25T14:16:49.983 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.982+0000 7ff27a7fc700 10 RGWWatcher::handle_notify() notify_id 592705487427 cookie 94831579522624 notifier 5005 bl.length()=636 2019-11-25T14:16:49.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.982+0000 7ff27a7fc700 10 cache put: name=default.rgw.meta+users.uid+bar.client.0 info.flags=0x17 2019-11-25T14:16:49.984 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:49.982+0000 7ff27a7fc700 10 moving default.rgw.meta+users.uid+bar.client.0 to cache LRU end 2019-11-25T14:16:50.140 INFO:teuthology.orchestra.run.smithi153.stdout:{ 2019-11-25T14:16:50.140 INFO:teuthology.orchestra.run.smithi153.stdout: "user_id": "bar.client.0", 2019-11-25T14:16:50.140 INFO:teuthology.orchestra.run.smithi153.stdout: "display_name": "Mr. bar.client.0 bar", 2019-11-25T14:16:50.140 INFO:teuthology.orchestra.run.smithi153.stdout: "email": "bar.client.0+test@test.test", 2019-11-25T14:16:50.140 INFO:teuthology.orchestra.run.smithi153.stdout: "suspended": 0, 2019-11-25T14:16:50.141 INFO:teuthology.orchestra.run.smithi153.stdout: "max_buckets": 1000, 2019-11-25T14:16:50.141 INFO:teuthology.orchestra.run.smithi153.stdout: "subusers": [ 2019-11-25T14:16:50.141 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T14:16:50.141 INFO:teuthology.orchestra.run.smithi153.stdout: "id": "bar.client.0:bar", 2019-11-25T14:16:50.141 INFO:teuthology.orchestra.run.smithi153.stdout: "permissions": "full-control" 2019-11-25T14:16:50.141 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T14:16:50.142 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T14:16:50.142 INFO:teuthology.orchestra.run.smithi153.stdout: "keys": [], 2019-11-25T14:16:50.142 INFO:teuthology.orchestra.run.smithi153.stdout: "swift_keys": [ 2019-11-25T14:16:50.142 INFO:teuthology.orchestra.run.smithi153.stdout: { 2019-11-25T14:16:50.142 INFO:teuthology.orchestra.run.smithi153.stdout: "user": "bar.client.0:bar", 2019-11-25T14:16:50.142 INFO:teuthology.orchestra.run.smithi153.stdout: "secret_key": "Oc+bjGv5QMe2VVR33uEMJY/0ElH/Y+VvxenR96n7sFgvLYpoWbJnjA==" 2019-11-25T14:16:50.142 INFO:teuthology.orchestra.run.smithi153.stdout: } 2019-11-25T14:16:50.143 INFO:teuthology.orchestra.run.smithi153.stdout: ], 2019-11-25T14:16:50.143 INFO:teuthology.orchestra.run.smithi153.stdout: "caps": [], 2019-11-25T14:16:50.143 INFO:teuthology.orchestra.run.smithi153.stdout: "op_mask": "read, write, delete", 2019-11-25T14:16:50.143 INFO:teuthology.orchestra.run.smithi153.stdout: "default_placement": "", 2019-11-25T14:16:50.143 INFO:teuthology.orchestra.run.smithi153.stdout: "default_storage_class": "", 2019-11-25T14:16:50.143 INFO:teuthology.orchestra.run.smithi153.stdout: "placement_tags": [], 2019-11-25T14:16:50.143 INFO:teuthology.orchestra.run.smithi153.stdout: "bucket_quota": { 2019-11-25T14:16:50.144 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T14:16:50.144 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T14:16:50.144 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T14:16:50.144 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T14:16:50.144 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T14:16:50.144 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T14:16:50.144 INFO:teuthology.orchestra.run.smithi153.stdout: "user_quota": { 2019-11-25T14:16:50.145 INFO:teuthology.orchestra.run.smithi153.stdout: "enabled": false, 2019-11-25T14:16:50.145 INFO:teuthology.orchestra.run.smithi153.stdout: "check_on_raw": false, 2019-11-25T14:16:50.145 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size": -1, 2019-11-25T14:16:50.145 INFO:teuthology.orchestra.run.smithi153.stdout: "max_size_kb": 0, 2019-11-25T14:16:50.145 INFO:teuthology.orchestra.run.smithi153.stdout: "max_objects": -1 2019-11-25T14:16:50.146 INFO:teuthology.orchestra.run.smithi153.stdout: }, 2019-11-25T14:16:50.146 INFO:teuthology.orchestra.run.smithi153.stdout: "temp_url_keys": [], 2019-11-25T14:16:50.146 INFO:teuthology.orchestra.run.smithi153.stdout: "type": "rgw", 2019-11-25T14:16:50.146 INFO:teuthology.orchestra.run.smithi153.stdout: "mfa_ids": [] 2019-11-25T14:16:50.146 INFO:teuthology.orchestra.run.smithi153.stdout:} 2019-11-25T14:16:50.146 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T14:16:50.179 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.178+0000 7ff2976a2ec0 20 remove_watcher() i=0 2019-11-25T14:16:50.179 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.178+0000 7ff2976a2ec0 2 removed watcher, disabling cache 2019-11-25T14:16:50.261 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.259+0000 7ff2976a2ec0 20 remove_watcher() i=1 2019-11-25T14:16:50.299 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.297+0000 7ff2976a2ec0 20 remove_watcher() i=2 2019-11-25T14:16:50.339 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.337+0000 7ff2976a2ec0 20 remove_watcher() i=3 2019-11-25T14:16:50.384 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.382+0000 7ff2976a2ec0 20 remove_watcher() i=4 2019-11-25T14:16:50.414 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.413+0000 7ff2976a2ec0 20 remove_watcher() i=5 2019-11-25T14:16:50.450 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.449+0000 7ff2976a2ec0 20 remove_watcher() i=6 2019-11-25T14:16:50.481 INFO:teuthology.orchestra.run.smithi153.stderr:2019-11-25T14:16:50.480+0000 7ff2976a2ec0 20 remove_watcher() i=7 2019-11-25T14:16:50.488 INFO:tasks.swift:Configuring testswift... 2019-11-25T14:16:50.489 INFO:teuthology.orchestra.run.smithi153:> cd /home/ubuntu/cephtest/swift && ./bootstrap 2019-11-25T14:16:53.183 INFO:teuthology.orchestra.run.smithi153.stdout:New python executable in /home/ubuntu/cephtest/swift/virtualenv/bin/python 2019-11-25T14:16:54.716 INFO:teuthology.orchestra.run.smithi153.stdout:Installing setuptools, pip, wheel...done. 2019-11-25T14:16:55.039 INFO:teuthology.orchestra.run.smithi153.stderr:Cache entry deserialization failed, entry ignored 2019-11-25T14:16:55.153 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting pip 2019-11-25T14:16:55.232 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/00/b6/9cfa56b4081ad13874b0c6f96af8ce16cfbc1cb06bedf8e9164ce5551ec1/pip-19.3.1-py2.py3-none-any.whl 2019-11-25T14:16:55.296 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: pip 2019-11-25T14:16:55.297 INFO:teuthology.orchestra.run.smithi153.stdout: Found existing installation: pip 9.0.1 2019-11-25T14:16:55.356 INFO:teuthology.orchestra.run.smithi153.stdout: Uninstalling pip-9.0.1: 2019-11-25T14:16:55.400 INFO:teuthology.orchestra.run.smithi153.stdout: Successfully uninstalled pip-9.0.1 2019-11-25T14:16:55.994 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed pip-19.3.1 2019-11-25T14:16:56.315 INFO:teuthology.orchestra.run.smithi153.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-25T14:16:57.045 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting setuptools==32.3.1 2019-11-25T14:16:57.050 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/69/19/b1dff551058ce79d88b1e3688f1c735590d7ddf44d10681512133b35019f/setuptools-32.3.1-py2.py3-none-any.whl 2019-11-25T14:16:57.106 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: setuptools 2019-11-25T14:16:57.106 INFO:teuthology.orchestra.run.smithi153.stdout: Found existing installation: setuptools 28.8.0 2019-11-25T14:16:57.123 INFO:teuthology.orchestra.run.smithi153.stdout: Uninstalling setuptools-28.8.0: 2019-11-25T14:16:57.132 INFO:teuthology.orchestra.run.smithi153.stdout: Successfully uninstalled setuptools-28.8.0 2019-11-25T14:16:57.290 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed setuptools-32.3.1 2019-11-25T14:16:57.835 INFO:teuthology.orchestra.run.smithi153.stderr:DEPRECATION: Python 2.7 will reach the end of its life on January 1st, 2020. Please upgrade your Python as Python 2.7 won't be maintained after that date. A future version of pip will drop support for Python 2.7. More details about Python 2 support in pip, can be found at https://pip.pypa.io/en/latest/development/release-process/#python-2-support 2019-11-25T14:16:58.032 INFO:teuthology.orchestra.run.smithi153.stdout:Processing /home/ubuntu/.cache/pip/wheels/d9/45/dd/65f0b38450c47cf7e5312883deb97d065e030c5cca0a365030/PyYAML-5.1.2-cp27-cp27mu-linux_x86_64.whl 2019-11-25T14:16:58.113 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting nose>=1.0.0 2019-11-25T14:16:58.116 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/99/4f/13fb671119e65c4dce97c60e67d3fd9e6f7f809f2b307e2611f4701205cb/nose-1.3.7-py2-none-any.whl 2019-11-25T14:16:58.263 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting boto>=2.0b4 2019-11-25T14:16:58.270 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/23/10/c0b78c27298029e4454a472a1919bde20cb182dab1662cec7f2ca1dcc523/boto-2.49.0-py2.py3-none-any.whl 2019-11-25T14:16:58.413 INFO:teuthology.orchestra.run.smithi153.stdout:Processing /home/ubuntu/.cache/pip/wheels/56/0f/19/fbbf81e5764e6d8b74501c4357a88c14c94466ec777c03734c/bunch-1.0.1-cp27-none-any.whl 2019-11-25T14:16:58.661 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting gevent>=0.14 2019-11-25T14:16:58.676 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/e6/e9/3a693414f706e12abe60554cd73c5ae8f848b182ae58018f93d86c9eb418/gevent-1.4.0-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-25T14:16:58.931 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting isodate>=0.4.4 2019-11-25T14:16:58.934 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/9b/9f/b36f7774ff5ea8e428fdcfc4bb332c39ee5b9362ddd3d40d9516a55221b2/isodate-0.6.0-py2.py3-none-any.whl 2019-11-25T14:16:59.003 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting eventlet>=0.9.14 2019-11-25T14:16:59.044 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/ef/01/83454d11bac9015f34e93cf11efcae169390ddf2df97cb73ca86de6465ed/eventlet-0.25.1-py2.py3-none-any.whl (222kB) 2019-11-25T14:16:59.236 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting netifaces>=0.5 2019-11-25T14:16:59.246 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/7e/02/ad1a92a72620cc17d448fe4dbdfbdf8fe1487ee7bfd82bb48308712c2f3c/netifaces-0.10.9-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-25T14:16:59.334 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting xattr<0.7.0,>=0.6 2019-11-25T14:16:59.345 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/c5/80/b25d549ae4bf4f3e9635a331b759ffca2de4dd8a78dc5106d1ca92f5d08d/xattr-0.6.4.tar.gz 2019-11-25T14:16:59.742 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting simplejson>=2.1.3 2019-11-25T14:16:59.753 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/98/87/a7b98aa9256c8843f92878966dc3d8d914c14aad97e2c5ce4798d5743e07/simplejson-3.17.0.tar.gz (83kB) 2019-11-25T14:17:00.093 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting WebOb==1.0 2019-11-25T14:17:00.105 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/bc/eb/91b8dd8f3a8d3c3943bf27a2d322501ad8b227bf29f59e792c2eae092bc9/WebOb-1.0.tar.gz (210kB) 2019-11-25T14:17:00.611 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting greenlet>=0.4.14; platform_python_implementation == "CPython" 2019-11-25T14:17:00.613 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/8b/6e/f2d25875713ad0885c8d3c69269697406652e6f64e1a6bd8264f7a609327/greenlet-0.4.15-cp27-cp27mu-manylinux1_x86_64.whl 2019-11-25T14:17:00.675 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting six 2019-11-25T14:17:00.676 INFO:teuthology.orchestra.run.smithi153.stdout: Using cached https://files.pythonhosted.org/packages/65/26/32b8464df2a97e6dd1b656ed26b2c194606c16fe163c695a992b36c11cdf/six-1.13.0-py2.py3-none-any.whl 2019-11-25T14:17:00.727 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting monotonic>=1.4 2019-11-25T14:17:00.737 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/ac/aa/063eca6a416f397bd99552c534c6d11d57f58f2e94c14780f3bbf818c4cf/monotonic-1.5-py2.py3-none-any.whl 2019-11-25T14:17:00.819 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting dnspython>=1.15.0 2019-11-25T14:17:00.829 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/ec/d3/3aa0e7213ef72b8585747aa0e271a9523e713813b9a20177ebe1e939deb0/dnspython-1.16.0-py2.py3-none-any.whl (188kB) 2019-11-25T14:17:00.967 INFO:teuthology.orchestra.run.smithi153.stdout:Collecting enum34; python_version < "3.4" 2019-11-25T14:17:00.979 INFO:teuthology.orchestra.run.smithi153.stdout: Downloading https://files.pythonhosted.org/packages/c5/db/e56e6b4bbac7c4a06de1c50de6fe1ef3810018ae11732a50f15f62c7d050/enum34-1.1.6-py2-none-any.whl 2019-11-25T14:17:01.018 INFO:teuthology.orchestra.run.smithi153.stdout:Building wheels for collected packages: xattr, simplejson, WebOb 2019-11-25T14:17:01.019 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for xattr (setup.py): started 2019-11-25T14:17:01.548 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for xattr (setup.py): finished with status 'done' 2019-11-25T14:17:01.552 INFO:teuthology.orchestra.run.smithi153.stdout: Created wheel for xattr: filename=xattr-0.6.4-cp27-cp27mu-linux_x86_64.whl size=24122 sha256=20ab06c2ebed24a07ec3dbf816ed1cadfb0e40cddb11867716bbd2c2f8b885d3 2019-11-25T14:17:01.552 INFO:teuthology.orchestra.run.smithi153.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/cb/2c/40/ea48345a21b521cdacebf61db13e3bf0c42c77ec664e3fe9e5 2019-11-25T14:17:01.554 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for simplejson (setup.py): started 2019-11-25T14:17:02.218 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:02:06.801 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.253 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:02:06.861 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.266 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:02:06.873 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.276 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:02:06.880 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.276 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:02:06.880 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.278 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:02:06.861 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.308 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:02:06.913 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.309 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:02:06.893 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.332 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:02:06.940 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:17:02.799 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for simplejson (setup.py): finished with status 'done' 2019-11-25T14:17:02.799 INFO:teuthology.orchestra.run.smithi153.stdout: Created wheel for simplejson: filename=simplejson-3.17.0-cp27-cp27mu-linux_x86_64.whl size=113127 sha256=dd00f7ea85d8ab8c719c50eba39394301aed6a74104b4d7d80923edd85c9b333 2019-11-25T14:17:02.799 INFO:teuthology.orchestra.run.smithi153.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/86/c0/83/dcd0339abb2640544bb8e0938aab2d069cef55e5647ce6e097 2019-11-25T14:17:02.817 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for WebOb (setup.py): started 2019-11-25T14:17:03.165 INFO:teuthology.orchestra.run.smithi153.stdout: Building wheel for WebOb (setup.py): finished with status 'done' 2019-11-25T14:17:03.165 INFO:teuthology.orchestra.run.smithi153.stdout: Created wheel for WebOb: filename=WebOb-1.0-cp27-none-any.whl size=58478 sha256=e0cb5da4641668ea2d7887b82d040b78729aec5564ac9270a946f32e4da18940 2019-11-25T14:17:03.165 INFO:teuthology.orchestra.run.smithi153.stdout: Stored in directory: /home/ubuntu/.cache/pip/wheels/30/41/c0/8ff9e679d72e05f355e6e2ec87d026131538c953a3b12496eb 2019-11-25T14:17:03.172 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully built xattr simplejson WebOb 2019-11-25T14:17:03.215 INFO:teuthology.orchestra.run.smithi153.stdout:Installing collected packages: PyYAML, nose, boto, bunch, greenlet, gevent, six, isodate, monotonic, dnspython, enum34, eventlet, netifaces, xattr, simplejson, WebOb 2019-11-25T14:17:04.670 INFO:teuthology.orchestra.run.smithi153.stdout:Successfully installed PyYAML-5.1.2 WebOb-1.0 boto-2.49.0 bunch-1.0.1 dnspython-1.16.0 enum34-1.1.6 eventlet-0.25.1 gevent-1.4.0 greenlet-0.4.15 isodate-0.6.0 monotonic-1.5 netifaces-0.10.9 nose-1.3.7 simplejson-3.17.0 six-1.13.0 xattr-0.6.4 2019-11-25T14:17:04.990 INFO:teuthology.orchestra.run.smithi153.stdout:running develop 2019-11-25T14:17:05.035 INFO:teuthology.orchestra.run.smithi153.stdout:running egg_info 2019-11-25T14:17:05.035 INFO:teuthology.orchestra.run.smithi153.stdout:creating swift.egg-info 2019-11-25T14:17:05.035 INFO:teuthology.orchestra.run.smithi153.stdout:writing swift.egg-info/PKG-INFO 2019-11-25T14:17:05.035 INFO:teuthology.orchestra.run.smithi153.stdout:writing top-level names to swift.egg-info/top_level.txt 2019-11-25T14:17:05.036 INFO:teuthology.orchestra.run.smithi153.stdout:writing dependency_links to swift.egg-info/dependency_links.txt 2019-11-25T14:17:05.039 INFO:teuthology.orchestra.run.smithi153.stdout:writing entry points to swift.egg-info/entry_points.txt 2019-11-25T14:17:05.039 INFO:teuthology.orchestra.run.smithi153.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-11-25T14:17:05.043 INFO:teuthology.orchestra.run.smithi153.stdout:reading manifest file 'swift.egg-info/SOURCES.txt' 2019-11-25T14:17:05.043 INFO:teuthology.orchestra.run.smithi153.stdout:reading manifest template 'MANIFEST.in' 2019-11-25T14:17:05.043 INFO:teuthology.orchestra.run.smithi153.stderr:warning: no files found matching 'ChangeLog' 2019-11-25T14:17:05.052 INFO:teuthology.orchestra.run.smithi153.stdout:writing manifest file 'swift.egg-info/SOURCES.txt' 2019-11-25T14:17:05.052 INFO:teuthology.orchestra.run.smithi153.stdout:running build_ext 2019-11-25T14:17:05.052 INFO:teuthology.orchestra.run.smithi153.stdout:Creating /home/ubuntu/cephtest/swift/virtualenv/lib/python2.7/site-packages/swift.egg-link (link to .) 2019-11-25T14:17:05.053 INFO:teuthology.orchestra.run.smithi153.stdout:Adding swift 1.4.4 to easy-install.pth file 2019-11-25T14:17:05.063 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.064 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-account-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.066 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-account-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.067 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-account-reaper script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.068 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-account-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.069 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-account-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.070 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-container-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.071 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-container-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.071 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-container-sync script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.072 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-container-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.073 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-container-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.074 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-drive-audit script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.075 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-get-nodes script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.076 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-init script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.077 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-object-auditor script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.078 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-object-expirer script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.079 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-object-info script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.080 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-object-replicator script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.081 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-object-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.082 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-object-updater script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.083 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-proxy-server script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.087 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-ring-builder script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.088 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-stats-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.093 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-stats-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.095 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-dispersion-populate script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.097 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-dispersion-report script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.098 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-bench script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.101 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-recon script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.102 INFO:teuthology.orchestra.run.smithi153.stdout:Installing swift-recon-cron script to /home/ubuntu/cephtest/swift/virtualenv/bin 2019-11-25T14:17:05.102 INFO:teuthology.orchestra.run.smithi153.stdout: 2019-11-25T14:17:05.103 INFO:teuthology.orchestra.run.smithi153.stdout:Installed /home/ubuntu/cephtest/swift 2019-11-25T14:17:05.103 INFO:teuthology.orchestra.run.smithi153.stdout:Processing dependencies for swift==1.4.4 2019-11-25T14:17:05.103 INFO:teuthology.orchestra.run.smithi153.stdout:Finished processing dependencies for swift==1.4.4 2019-11-25T14:17:05.114 INFO:teuthology.orchestra.run.smithi153:> cat > /home/ubuntu/cephtest/archive/testswift.client.0.conf 2019-11-25T14:17:05.166 INFO:teuthology.orchestra.run.smithi153:> SWIFT_TEST_CONFIG_FILE=/home/ubuntu/cephtest/archive/testswift.client.0.conf /home/ubuntu/cephtest/swift/virtualenv/bin/nosetests -w /home/ubuntu/cephtest/swift/test/functional -v -a '!fails_on_rgw' 2019-11-25T14:17:33.835 INFO:teuthology.orchestra.run.smithi153.stderr:testAccountHead (test.functional.tests.TestAccount) ... ok 2019-11-25T14:17:49.807 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerListing (test.functional.tests.TestAccount) ... ok 2019-11-25T14:20:38.087 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:05:42.670 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:20:38.131 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:05:42.714 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:20:38.134 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:05:42.741 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:20:38.139 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:05:42.746 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:20:38.141 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:05:42.748 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:20:38.144 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:05:42.751 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:20:38.268 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:05:42.876 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:20:38.270 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:05:42.877 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:20:38.328 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:05:42.911 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:39.835 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerSerializedInfo (test.functional.tests.TestAccount) ... ok 2019-11-25T14:21:56.999 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:07:01.582 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.025 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:07:01.608 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.032 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:07:01.640 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.037 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:07:01.644 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.043 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:07:01.650 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.043 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:07:01.650 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.155 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:07:01.738 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.193 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:07:01.800 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.193 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:07:01.801 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:21:57.388 INFO:teuthology.orchestra.run.smithi153.stderr:testContainersOrderedByName (test.functional.tests.TestAccount) ... ok 2019-11-25T14:21:59.574 INFO:teuthology.orchestra.run.smithi153.stderr:testInvalidAuthToken (test.functional.tests.TestAccount) ... ok 2019-11-25T14:22:19.326 INFO:teuthology.orchestra.run.smithi153.stderr:testLastContainerMarker (test.functional.tests.TestAccount) ... ok 2019-11-25T14:22:47.125 INFO:teuthology.orchestra.run.smithi153.stderr:testListingLimit (test.functional.tests.TestAccount) ... ok 2019-11-25T14:23:44.939 INFO:teuthology.orchestra.run.smithi153.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccount) ... ok 2019-11-25T14:23:45.187 INFO:teuthology.orchestra.run.smithi153.stderr:testPUT (test.functional.tests.TestAccount) ... ok 2019-11-25T14:25:24.291 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:10:28.874 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:25:24.302 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:10:28.886 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:25:24.341 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:10:28.949 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:25:24.344 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:10:28.952 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:25:24.390 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:10:28.997 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:25:24.392 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:10:29.000 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:25:24.480 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:10:29.087 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:25:24.484 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:10:29.092 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:25:24.520 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:10:29.103 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.727 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:12:14.310 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.733 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:12:14.317 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.744 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:12:14.351 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.745 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:12:14.353 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.749 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:12:14.357 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.761 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:12:14.368 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.818 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:12:14.401 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.836 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:12:14.444 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:27:09.841 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:12:14.449 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:29:38.646 INFO:teuthology.orchestra.run.smithi153.stderr:testGetRequest (test.functional.tests.TestAccountNoContainers) ... ok 2019-11-25T14:29:41.083 INFO:teuthology.orchestra.run.smithi153.stderr:testGetRequest (test.functional.tests.TestAccountNoContainersUTF8) ... ok 2019-11-25T14:30:18.326 INFO:teuthology.orchestra.run.smithi153.stderr:testAccountHead (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:30:37.206 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerListing (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:32:21.741 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:17:26.324 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:32:21.751 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:17:26.359 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:32:21.755 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:17:26.363 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:32:21.759 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:17:26.342 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:32:21.760 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:17:26.368 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:32:21.764 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:17:26.371 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:32:21.939 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:17:26.547 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:32:21.955 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:17:26.563 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:32:22.028 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:17:26.610 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.145 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:18:50.728 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.150 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:18:50.734 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.217 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:18:50.825 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.224 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:18:50.832 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.224 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:18:50.808 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.232 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:18:50.840 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.239 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:18:50.847 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.310 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:18:50.917 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:33:46.313 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:18:50.921 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:35:00.567 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerSerializedInfo (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:35:19.311 INFO:teuthology.orchestra.run.smithi153.stderr:testContainersOrderedByName (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:35:21.114 INFO:teuthology.orchestra.run.smithi153.stderr:testInvalidAuthToken (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:35:41.504 INFO:teuthology.orchestra.run.smithi153.stderr:testLastContainerMarker (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:36:06.675 INFO:teuthology.orchestra.run.smithi153.stderr:testListingLimit (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:37:33.114 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:22:37.697 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:33.126 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:22:37.710 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:33.134 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:22:37.742 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:33.138 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:22:37.746 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:33.173 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:22:37.781 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:33.178 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:22:37.786 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:33.320 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:22:37.928 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:33.334 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:22:37.942 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:33.343 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:22:37.927 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:37:51.973 INFO:teuthology.orchestra.run.smithi153.stderr:testMarkerLimitContainerList (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:37:52.209 INFO:teuthology.orchestra.run.smithi153.stderr:testPUT (test.functional.tests.TestAccountUTF8) ... ok 2019-11-25T14:42:07.881 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:27:12.464 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:07.915 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:27:12.499 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:07.930 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:27:12.536 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:07.935 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:27:12.543 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:07.941 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:27:12.549 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:07.945 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:27:12.553 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:08.059 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:27:12.667 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:08.067 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:27:12.651 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:08.074 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:27:12.682 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.273 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:27:47.856 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.282 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:27:47.866 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.287 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:27:47.895 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.290 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:27:47.898 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.292 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:27:47.901 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.294 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:27:47.902 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.324 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:27:47.908 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.352 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:27:47.960 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:42:43.356 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:27:47.964 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:44:48.594 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainer) ... ok 2019-11-25T14:44:53.206 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerFileList (test.functional.tests.TestContainer) ... ok 2019-11-25T14:44:54.982 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-11-25T14:44:58.273 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainer) ... ok 2019-11-25T14:44:59.144 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerInfo (test.functional.tests.TestContainer) ... ok 2019-11-25T14:44:59.337 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-11-25T14:45:10.615 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerNameLimit (test.functional.tests.TestContainer) ... ok 2019-11-25T14:45:17.365 INFO:teuthology.orchestra.run.smithi153.stderr:testCreate (test.functional.tests.TestContainer) ... ok 2019-11-25T14:45:30.287 INFO:teuthology.orchestra.run.smithi153.stderr:testDelete (test.functional.tests.TestContainer) ... ok 2019-11-25T14:45:30.877 INFO:teuthology.orchestra.run.smithi153.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-11-25T14:45:37.740 INFO:teuthology.orchestra.run.smithi153.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainer) ... ok 2019-11-25T14:45:38.277 INFO:teuthology.orchestra.run.smithi153.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainer) ... ok 2019-11-25T14:47:18.506 INFO:teuthology.orchestra.run.smithi153.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainer) ... ok 2019-11-25T14:47:21.389 INFO:teuthology.orchestra.run.smithi153.stderr:testFileOrder (test.functional.tests.TestContainer) ... ok 2019-11-25T14:47:35.783 INFO:teuthology.orchestra.run.smithi153.stderr:testFileThenContainerDelete (test.functional.tests.TestContainer) ... ok 2019-11-25T14:47:41.577 INFO:teuthology.orchestra.run.smithi153.stderr:testLastFileMarker (test.functional.tests.TestContainer) ... ok 2019-11-25T14:47:55.333 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:32:59.917 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:47:55.336 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:32:59.920 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:47:55.343 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:32:59.951 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:47:55.347 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:32:59.953 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:47:55.349 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:32:59.957 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:47:55.349 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:32:59.957 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:47:55.533 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:33:00.141 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:47:55.534 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:33:00.141 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:47:55.558 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:33:00.142 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:48:19.011 INFO:teuthology.orchestra.run.smithi153.stderr:testMarkerLimitFileList (test.functional.tests.TestContainer) ... ok 2019-11-25T14:49:27.424 INFO:teuthology.orchestra.run.smithi153.stderr:testPrefixAndLimit (test.functional.tests.TestContainer) ... ok 2019-11-25T14:49:27.485 INFO:teuthology.orchestra.run.smithi153.stderr:testTooLongName (test.functional.tests.TestContainer) ... ok 2019-11-25T14:50:38.094 INFO:teuthology.orchestra.run.smithi153.stderr:testUnorderedList (test.functional.tests.TestContainer) ... ok 2019-11-25T14:53:05.182 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:38:09.765 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:53:05.188 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:38:09.772 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:53:05.197 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:38:09.805 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:53:05.200 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:38:09.808 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:53:05.201 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:38:09.809 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:53:05.208 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:38:09.816 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:53:05.391 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:38:09.975 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:53:05.454 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:38:10.061 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:53:05.456 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:38:10.064 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:55:57.608 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerListing (test.functional.tests.TestContainerPaths) ... ok 2019-11-25T14:56:12.512 INFO:teuthology.orchestra.run.smithi153.stderr:testStructure (test.functional.tests.TestContainerPaths) ... ok 2019-11-25T14:56:31.545 INFO:teuthology.orchestra.run.smithi153.stderr:testTraverseContainer (test.functional.tests.TestContainerPaths) ... ok 2019-11-25T14:58:08.123 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerExistenceCachingProblem (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:11.162 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerFileList (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:12.292 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerFileListOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:15.656 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerFileListWithLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:15.979 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerInfo (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:16.392 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerInfoOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:16.818 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:43:21.402 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:16.828 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:43:21.412 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:16.837 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:43:21.445 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:16.840 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:43:21.448 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:16.842 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:43:21.451 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:16.847 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:43:21.455 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:17.024 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:43:21.632 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:17.029 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:43:21.613 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:17.066 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:43:21.674 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:23.750 INFO:teuthology.orchestra.run.smithi153.stderr:testContainerNameLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:31.518 INFO:teuthology.orchestra.run.smithi153.stderr:testCreate (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:45.428 INFO:teuthology.orchestra.run.smithi153.stderr:testDelete (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:45.730 INFO:teuthology.orchestra.run.smithi153.stderr:testDeleteOnContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:53.245 INFO:teuthology.orchestra.run.smithi153.stderr:testDeleteOnContainerWithFiles (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:53.582 INFO:teuthology.orchestra.run.smithi153.stderr:testFileCreateInContainerThatDoesNotExist (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T14:58:56.391 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:44:00.974 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:56.394 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:44:00.978 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:56.432 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:44:01.016 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:56.458 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:44:01.066 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:56.461 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:44:01.069 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:56.461 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:44:01.069 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:56.468 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:44:01.076 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:56.497 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:44:01.105 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T14:58:56.511 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:44:01.119 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:00:33.479 INFO:teuthology.orchestra.run.smithi153.stderr:testFileListingLimitMarkerPrefix (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T15:00:36.708 INFO:teuthology.orchestra.run.smithi153.stderr:testFileOrder (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T15:00:53.991 INFO:teuthology.orchestra.run.smithi153.stderr:testFileThenContainerDelete (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T15:01:00.537 INFO:teuthology.orchestra.run.smithi153.stderr:testLastFileMarker (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T15:01:07.269 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:46:11.852 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:07.274 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:46:11.858 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:07.342 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:46:11.950 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:07.369 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:46:11.977 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:07.377 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:46:11.985 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:07.383 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:46:11.991 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:07.406 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:46:11.989 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:07.489 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:46:12.097 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:07.489 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:46:12.097 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:01:42.211 INFO:teuthology.orchestra.run.smithi153.stderr:testMarkerLimitFileList (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T15:02:53.012 INFO:teuthology.orchestra.run.smithi153.stderr:testPrefixAndLimit (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T15:02:53.203 INFO:teuthology.orchestra.run.smithi153.stderr:testTooLongName (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T15:03:26.062 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:48:30.645 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:03:26.067 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:48:30.651 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:03:26.075 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:48:30.683 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:03:26.081 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:48:30.689 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:03:26.084 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:48:30.692 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:03:26.089 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:48:30.697 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:03:26.176 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:48:30.760 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:03:26.181 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:48:30.789 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:03:26.188 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:48:30.795 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:04:06.277 INFO:teuthology.orchestra.run.smithi153.stderr:testUnorderedList (test.functional.tests.TestContainerUTF8) ... ok 2019-11-25T15:07:18.545 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:52:23.129 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:07:18.550 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:52:23.133 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:07:18.571 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:52:23.178 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:07:18.613 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:52:23.221 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:07:18.641 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:52:23.248 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:07:18.642 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:52:23.250 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:07:18.690 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:52:23.274 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:07:18.738 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:52:23.346 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:07:18.739 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:52:23.347 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:36.444 INFO:teuthology.orchestra.run.smithi153.stderr:test_get_manifest (test.functional.tests.TestDlo) ... ok 2019-11-25T15:08:36.841 INFO:teuthology.orchestra.run.smithi153.stderr:test_get_manifest_document_itself (test.functional.tests.TestDlo) ... ok 2019-11-25T15:08:36.999 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:53:41.583 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:37.007 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:53:41.591 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:37.017 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:53:41.625 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:37.026 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:53:41.633 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:37.050 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:53:41.657 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:37.055 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:53:41.663 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:37.082 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:53:41.666 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:37.141 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:53:41.749 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:37.143 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:53:41.750 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:08:44.247 INFO:teuthology.orchestra.run.smithi153.stderr:test_get_range (test.functional.tests.TestDlo) ... ok 2019-11-25T15:08:45.588 INFO:teuthology.orchestra.run.smithi153.stderr:test_get_range_out_of_range (test.functional.tests.TestDlo) ... ok 2019-11-25T15:09:55.829 INFO:teuthology.orchestra.run.smithi153.stderr:test_get_manifest (test.functional.tests.TestDloUTF8) ... ok 2019-11-25T15:09:56.585 INFO:teuthology.orchestra.run.smithi153.stderr:test_get_manifest_document_itself (test.functional.tests.TestDloUTF8) ... ok 2019-11-25T15:10:05.317 INFO:teuthology.orchestra.run.smithi153.stderr:test_get_range (test.functional.tests.TestDloUTF8) ... ok 2019-11-25T15:10:06.505 INFO:teuthology.orchestra.run.smithi153.stderr:test_get_range_out_of_range (test.functional.tests.TestDloUTF8) ... ok 2019-11-25T15:10:42.364 INFO:teuthology.orchestra.run.smithi153.stderr:testBadHeaders (test.functional.tests.TestFile) ... ok 2019-11-25T15:10:42.754 INFO:teuthology.orchestra.run.smithi153.stderr:testBlankMetadataName (test.functional.tests.TestFile) ... ok 2019-11-25T15:13:48.374 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:58:52.958 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:13:48.388 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:58:52.972 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:13:48.401 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:58:53.009 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:13:48.405 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:58:53.013 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:13:48.406 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:58:53.014 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:13:48.410 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:58:53.018 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:13:48.593 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:05:58:53.201 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:13:48.602 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:05:58:53.210 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:13:48.629 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:05:58:53.213 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.457 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:00:50.040 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.459 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:00:50.043 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.496 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:00:50.102 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.496 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:00:50.104 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.508 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:00:50.116 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.510 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:00:50.118 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.570 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:00:50.177 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.575 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:00:50.159 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:15:45.578 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:00:50.186 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:38.342 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:18:38.336+0000 16949700 -1 osd.2 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:57.740723+0000 front 2019-11-25T15:17:56.988949+0000 (oldest deadline 2019-11-25T15:18:37.899636+0000) 2019-11-25T15:18:38.869 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:18:38.843+0000 16949700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:57.030832+0000 front 2019-11-25T15:17:57.070694+0000 (oldest deadline 2019-11-25T15:18:38.040595+0000) 2019-11-25T15:18:39.312 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:18:39.308+0000 16949700 -1 osd.2 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:57.740723+0000 front 2019-11-25T15:17:56.988949+0000 (oldest deadline 2019-11-25T15:18:37.899636+0000) 2019-11-25T15:18:39.465 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:18:39.456+0000 16949700 -1 osd.1 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:53.812214+0000 front 2019-11-25T15:17:54.388224+0000 (oldest deadline 2019-11-25T15:18:38.472283+0000) 2019-11-25T15:18:39.864 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:18:39.860+0000 16949700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:57.030832+0000 front 2019-11-25T15:17:57.070694+0000 (oldest deadline 2019-11-25T15:18:38.040595+0000) 2019-11-25T15:18:40.235 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:18:40.224+0000 16949700 -1 osd.4 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:57.798164+0000 front 2019-11-25T15:17:57.832334+0000 (oldest deadline 2019-11-25T15:18:40.065016+0000) 2019-11-25T15:18:40.292 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:18:40.289+0000 16949700 -1 osd.2 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:57.740723+0000 front 2019-11-25T15:17:56.988949+0000 (oldest deadline 2019-11-25T15:18:37.899636+0000) 2019-11-25T15:18:40.379 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:18:40.371+0000 16949700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:54.989771+0000 front 2019-11-25T15:17:57.736123+0000 (oldest deadline 2019-11-25T15:18:39.496104+0000) 2019-11-25T15:18:40.471 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:18:40.463+0000 16949700 -1 osd.1 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:53.812214+0000 front 2019-11-25T15:17:54.388224+0000 (oldest deadline 2019-11-25T15:18:38.472283+0000) 2019-11-25T15:18:40.921 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:18:40.917+0000 16949700 -1 osd.6 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:57.030832+0000 front 2019-11-25T15:17:57.070694+0000 (oldest deadline 2019-11-25T15:18:38.040595+0000) 2019-11-25T15:18:41.265 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:18:41.258+0000 16949700 -1 osd.2 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:57.740723+0000 front 2019-11-25T15:17:56.988949+0000 (oldest deadline 2019-11-25T15:18:37.899636+0000) 2019-11-25T15:18:41.339 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:18:41.337+0000 16949700 -1 osd.0 138 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:17:54.989771+0000 front 2019-11-25T15:17:57.736123+0000 (oldest deadline 2019-11-25T15:18:39.496104+0000) 2019-11-25T15:18:48.442 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:03:53.025 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:48.449 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:03:53.033 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:48.466 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:03:53.073 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:48.469 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:03:53.077 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:48.470 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:03:53.078 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:48.475 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:03:53.083 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:48.569 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:03:53.152 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:48.606 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:03:53.213 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:18:48.622 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:03:53.230 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:19:01.845 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:01.816+0000 28256700 -1 reset not still connected to 0x3c72e3d0 2019-11-25T15:19:36.225 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:36.221+0000 16949700 -1 osd.3 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:19:37.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:37.201+0000 16949700 -1 osd.3 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:19:38.200 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:38.198+0000 16949700 -1 osd.3 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:19:39.234 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:39.223+0000 16949700 -1 osd.3 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:19:39.710 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:19:39.697+0000 16949700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.034475+0000 front 2019-11-25T15:18:58.035964+0000 (oldest deadline 2019-11-25T15:19:39.270779+0000) 2019-11-25T15:19:40.278 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:40.275+0000 16949700 -1 osd.3 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:19:40.715 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:19:40.712+0000 16949700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.034475+0000 front 2019-11-25T15:18:58.035964+0000 (oldest deadline 2019-11-25T15:19:39.270779+0000) 2019-11-25T15:19:41.264 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:41.261+0000 16949700 -1 osd.3 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:19:41.739 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:19:41.736+0000 16949700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.034475+0000 front 2019-11-25T15:18:58.035964+0000 (oldest deadline 2019-11-25T15:19:39.270779+0000) 2019-11-25T15:19:42.140 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:19:42.137+0000 16949700 -1 osd.2 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.688469+0000 front 2019-11-25T15:18:58.669395+0000 (oldest deadline 2019-11-25T15:19:41.346858+0000) 2019-11-25T15:19:42.229 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:42.227+0000 16949700 -1 osd.3 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:19:42.695 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:19:42.685+0000 16949700 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.000688+0000 front 2019-11-25T15:18:57.794593+0000 (oldest deadline 2019-11-25T15:19:41.874117+0000) 2019-11-25T15:19:42.716 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:19:42.712+0000 16949700 -1 osd.4 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:59.011100+0000 front 2019-11-25T15:18:59.009714+0000 (oldest deadline 2019-11-25T15:19:41.942351+0000) 2019-11-25T15:19:42.791 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:19:42.788+0000 16949700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.034475+0000 front 2019-11-25T15:18:58.035964+0000 (oldest deadline 2019-11-25T15:19:39.270779+0000) 2019-11-25T15:19:42.894 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:19:42.890+0000 16949700 -1 osd.1 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:57.470304+0000 front 2019-11-25T15:18:57.821614+0000 (oldest deadline 2019-11-25T15:19:41.942802+0000) 2019-11-25T15:19:43.180 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:19:43.176+0000 16949700 -1 osd.2 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.688469+0000 front 2019-11-25T15:18:58.669395+0000 (oldest deadline 2019-11-25T15:19:41.346858+0000) 2019-11-25T15:19:43.188 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:43.185+0000 16949700 -1 osd.3 144 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:19:43.702 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:19:43.698+0000 16949700 -1 osd.4 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:59.011100+0000 front 2019-11-25T15:18:59.009714+0000 (oldest deadline 2019-11-25T15:19:41.942351+0000) 2019-11-25T15:19:43.709 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:19:43.705+0000 16949700 -1 osd.5 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.000688+0000 front 2019-11-25T15:18:57.794593+0000 (oldest deadline 2019-11-25T15:19:41.874117+0000) 2019-11-25T15:19:43.808 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:19:43.806+0000 16949700 -1 osd.0 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.034475+0000 front 2019-11-25T15:18:58.035964+0000 (oldest deadline 2019-11-25T15:19:39.270779+0000) 2019-11-25T15:19:43.866 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:19:43.864+0000 16949700 -1 osd.1 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:57.470304+0000 front 2019-11-25T15:18:57.821614+0000 (oldest deadline 2019-11-25T15:19:41.942802+0000) 2019-11-25T15:19:44.166 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:19:44.163+0000 16949700 -1 osd.2 144 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:18:58.688469+0000 front 2019-11-25T15:18:58.669395+0000 (oldest deadline 2019-11-25T15:19:41.346858+0000) 2019-11-25T15:19:44.588 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:19:44.571+0000 16949700 -1 osd.3 145 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e142) 2019-11-25T15:20:55.892 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:20:55.881+0000 33a6d700 -1 osd.6 149 build_incremental_map_msg missing incremental map 149 2019-11-25T15:20:56.112 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:20:56.098+0000 33a6d700 -1 osd.6 149 build_incremental_map_msg missing incremental map 149 2019-11-25T15:20:56.170 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:20:56.107+0000 33a6d700 -1 osd.6 149 build_incremental_map_msg missing incremental map 149 2019-11-25T15:20:56.232 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:20:56.167+0000 39278700 -1 osd.6 149 build_incremental_map_msg missing incremental map 149 2019-11-25T15:20:56.296 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:20:56.212+0000 34a6f700 -1 osd.6 149 build_incremental_map_msg missing incremental map 149 2019-11-25T15:20:56.893 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:20:56.779+0000 11ca9700 -1 osd.6 149 build_incremental_map_msg missing incremental map 149 2019-11-25T15:22:01.278 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:22:01.275+0000 16949700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:16.428384+0000 front 2019-11-25T15:21:16.404527+0000 (oldest deadline 2019-11-25T15:22:01.070211+0000) 2019-11-25T15:22:02.289 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:22:02.284+0000 16949700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:16.428384+0000 front 2019-11-25T15:21:16.404527+0000 (oldest deadline 2019-11-25T15:22:01.070211+0000) 2019-11-25T15:22:03.166 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:22:03.163+0000 16949700 -1 osd.0 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:17.044844+0000 front 2019-11-25T15:21:17.049896+0000 (oldest deadline 2019-11-25T15:22:02.785857+0000) 2019-11-25T15:22:03.173 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:22:03.171+0000 16949700 -1 osd.1 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:21.057102+0000 front 2019-11-25T15:21:21.012779+0000 (oldest deadline 2019-11-25T15:22:02.943649+0000) 2019-11-25T15:22:03.292 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:22:03.287+0000 16949700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:16.428384+0000 front 2019-11-25T15:21:16.404527+0000 (oldest deadline 2019-11-25T15:22:01.070211+0000) 2019-11-25T15:22:03.656 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:22:03.635+0000 16949700 -1 osd.5 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:18.947271+0000 front 2019-11-25T15:21:18.565812+0000 (oldest deadline 2019-11-25T15:22:03.238415+0000) 2019-11-25T15:22:03.875 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:22:03.863+0000 16949700 -1 osd.7 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:20.025864+0000 front 2019-11-25T15:21:21.002616+0000 (oldest deadline 2019-11-25T15:22:02.819748+0000) 2019-11-25T15:22:04.174 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:22:04.170+0000 16949700 -1 osd.1 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:21.057102+0000 front 2019-11-25T15:21:21.012779+0000 (oldest deadline 2019-11-25T15:22:02.943649+0000) 2019-11-25T15:22:04.204 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:22:04.197+0000 16949700 -1 osd.0 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:17.044844+0000 front 2019-11-25T15:21:17.049896+0000 (oldest deadline 2019-11-25T15:22:02.785857+0000) 2019-11-25T15:22:04.338 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:22:04.330+0000 16949700 -1 osd.4 149 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:21:16.428384+0000 front 2019-11-25T15:21:16.404527+0000 (oldest deadline 2019-11-25T15:22:01.070211+0000) 2019-11-25T15:23:04.633 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:08:09.217 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:04.642 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:08:09.226 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:04.651 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:08:09.258 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:04.652 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:08:09.259 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:04.655 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:08:09.262 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:04.657 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:08:09.265 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:04.878 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:08:09.485 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:04.901 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:08:09.485 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:04.903 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:08:09.510 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:23:15.224 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:23:15.168+0000 38276700 -1 osd.6 153 build_incremental_map_msg missing incremental map 153 2019-11-25T15:23:15.496 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:23:15.447+0000 3aa7b700 -1 osd.6 153 build_incremental_map_msg missing incremental map 153 2019-11-25T15:23:15.826 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:23:15.678+0000 36272700 -1 osd.6 153 build_incremental_map_msg missing incremental map 153 2019-11-25T15:23:15.983 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:23:15.770+0000 3426e700 -1 osd.6 153 build_incremental_map_msg missing incremental map 153 2019-11-25T15:23:16.052 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:23:15.954+0000 38a77700 -1 osd.6 153 build_incremental_map_msg missing incremental map 153 2019-11-25T15:23:46.466 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:46.460+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:46.468 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:46.466+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:47.495 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:47.491+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:47.495 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:47.493+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:47.627 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:47.624+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:48.459 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:48.456+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:48.460 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:48.457+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:48.566 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:23:48.561+0000 16949700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.783191+0000 front 2019-11-25T15:23:04.786059+0000 (oldest deadline 2019-11-25T15:23:48.062091+0000) 2019-11-25T15:23:48.596 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:48.592+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:48.689 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:23:48.686+0000 16949700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.040852+0000 front 2019-11-25T15:23:05.323246+0000 (oldest deadline 2019-11-25T15:23:47.865707+0000) 2019-11-25T15:23:49.263 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:23:49.259+0000 3426e700 -1 osd.6 155 build_incremental_map_msg missing incremental map 149 2019-11-25T15:23:49.263 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:23:49.260+0000 3426e700 -1 osd.6 155 build_incremental_map_msg missing incremental map 153 2019-11-25T15:23:49.424 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:49.420+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:49.425 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:49.422+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:49.560 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:49.558+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:49.586 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:23:49.581+0000 16949700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.783191+0000 front 2019-11-25T15:23:04.786059+0000 (oldest deadline 2019-11-25T15:23:48.062091+0000) 2019-11-25T15:23:49.679 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:23:49.676+0000 16949700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.040852+0000 front 2019-11-25T15:23:05.323246+0000 (oldest deadline 2019-11-25T15:23:47.865707+0000) 2019-11-25T15:23:49.691 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:23:49.687+0000 16949700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.666384+0000 front 2019-11-25T15:23:05.627927+0000 (oldest deadline 2019-11-25T15:23:48.858540+0000) 2019-11-25T15:23:49.931 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:23:49.921+0000 16949700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.560664+0000 front 2019-11-25T15:23:05.578412+0000 (oldest deadline 2019-11-25T15:23:49.534138+0000) 2019-11-25T15:23:50.220 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:23:50.215+0000 16949700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.636997+0000 front 2019-11-25T15:23:04.787590+0000 (oldest deadline 2019-11-25T15:23:49.401530+0000) 2019-11-25T15:23:50.427 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:50.418+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:50.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:50.430+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:50.521 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:50.518+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:50.584 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:23:50.581+0000 16949700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.783191+0000 front 2019-11-25T15:23:04.786059+0000 (oldest deadline 2019-11-25T15:23:48.062091+0000) 2019-11-25T15:23:50.666 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:23:50.663+0000 16949700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.040852+0000 front 2019-11-25T15:23:05.323246+0000 (oldest deadline 2019-11-25T15:23:47.865707+0000) 2019-11-25T15:23:50.705 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:23:50.702+0000 16949700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.666384+0000 front 2019-11-25T15:23:05.627927+0000 (oldest deadline 2019-11-25T15:23:48.858540+0000) 2019-11-25T15:23:50.886 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:23:50.882+0000 16949700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.560664+0000 front 2019-11-25T15:23:05.578412+0000 (oldest deadline 2019-11-25T15:23:49.534138+0000) 2019-11-25T15:23:51.227 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:23:51.223+0000 16949700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.636997+0000 front 2019-11-25T15:23:04.787590+0000 (oldest deadline 2019-11-25T15:23:49.401530+0000) 2019-11-25T15:23:51.387 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:51.384+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:51.388 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:51.385+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:51.513 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:51.509+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:51.591 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:23:51.588+0000 16949700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.783191+0000 front 2019-11-25T15:23:04.786059+0000 (oldest deadline 2019-11-25T15:23:48.062091+0000) 2019-11-25T15:23:51.675 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:23:51.672+0000 16949700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.040852+0000 front 2019-11-25T15:23:05.323246+0000 (oldest deadline 2019-11-25T15:23:47.865707+0000) 2019-11-25T15:23:51.712 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:23:51.708+0000 16949700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.666384+0000 front 2019-11-25T15:23:05.627927+0000 (oldest deadline 2019-11-25T15:23:48.858540+0000) 2019-11-25T15:23:51.910 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:23:51.906+0000 16949700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.560664+0000 front 2019-11-25T15:23:05.578412+0000 (oldest deadline 2019-11-25T15:23:49.534138+0000) 2019-11-25T15:23:52.188 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:23:52.185+0000 16949700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.636997+0000 front 2019-11-25T15:23:04.787590+0000 (oldest deadline 2019-11-25T15:23:49.401530+0000) 2019-11-25T15:23:52.383 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:52.380+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:52.384 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:52.381+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:52.544 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:52.540+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:52.572 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:23:52.570+0000 16949700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.783191+0000 front 2019-11-25T15:23:04.786059+0000 (oldest deadline 2019-11-25T15:23:48.062091+0000) 2019-11-25T15:23:52.646 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:23:52.643+0000 16949700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.040852+0000 front 2019-11-25T15:23:05.323246+0000 (oldest deadline 2019-11-25T15:23:47.865707+0000) 2019-11-25T15:23:52.733 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:23:52.729+0000 16949700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.666384+0000 front 2019-11-25T15:23:05.627927+0000 (oldest deadline 2019-11-25T15:23:48.858540+0000) 2019-11-25T15:23:52.919 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:23:52.915+0000 16949700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.560664+0000 front 2019-11-25T15:23:05.578412+0000 (oldest deadline 2019-11-25T15:23:49.534138+0000) 2019-11-25T15:23:53.210 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:23:53.206+0000 16949700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.636997+0000 front 2019-11-25T15:23:04.787590+0000 (oldest deadline 2019-11-25T15:23:49.401530+0000) 2019-11-25T15:23:53.413 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:53.411+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:53.415 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:53.413+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:53.516 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:53.513+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:53.531 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:23:53.528+0000 16949700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.783191+0000 front 2019-11-25T15:23:04.786059+0000 (oldest deadline 2019-11-25T15:23:48.062091+0000) 2019-11-25T15:23:53.636 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:23:53.632+0000 16949700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.040852+0000 front 2019-11-25T15:23:05.323246+0000 (oldest deadline 2019-11-25T15:23:47.865707+0000) 2019-11-25T15:23:53.780 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:23:53.776+0000 16949700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.666384+0000 front 2019-11-25T15:23:05.627927+0000 (oldest deadline 2019-11-25T15:23:48.858540+0000) 2019-11-25T15:23:53.971 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:23:53.965+0000 16949700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.560664+0000 front 2019-11-25T15:23:05.578412+0000 (oldest deadline 2019-11-25T15:23:49.534138+0000) 2019-11-25T15:23:54.223 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:23:54.219+0000 16949700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.636997+0000 front 2019-11-25T15:23:04.787590+0000 (oldest deadline 2019-11-25T15:23:49.401530+0000) 2019-11-25T15:23:54.405 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:54.402+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:54.405 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:54.403+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:54.494 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:23:54.491+0000 16949700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.783191+0000 front 2019-11-25T15:23:04.786059+0000 (oldest deadline 2019-11-25T15:23:48.062091+0000) 2019-11-25T15:23:54.526 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:54.523+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:54.595 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:23:54.591+0000 16949700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.040852+0000 front 2019-11-25T15:23:05.323246+0000 (oldest deadline 2019-11-25T15:23:47.865707+0000) 2019-11-25T15:23:54.789 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:23:54.784+0000 16949700 -1 osd.4 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.666384+0000 front 2019-11-25T15:23:05.627927+0000 (oldest deadline 2019-11-25T15:23:48.858540+0000) 2019-11-25T15:23:54.978 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:23:54.973+0000 16949700 -1 osd.5 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.560664+0000 front 2019-11-25T15:23:05.578412+0000 (oldest deadline 2019-11-25T15:23:49.534138+0000) 2019-11-25T15:23:55.248 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:23:55.235+0000 16949700 -1 osd.7 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.636997+0000 front 2019-11-25T15:23:04.787590+0000 (oldest deadline 2019-11-25T15:23:49.401530+0000) 2019-11-25T15:23:55.440 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:55.436+0000 16949700 -1 osd.3 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:06.095853+0000 front 2019-11-25T15:23:05.688034+0000 (oldest deadline 2019-11-25T15:23:46.382077+0000) 2019-11-25T15:23:55.441 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:23:55.438+0000 16949700 -1 osd.3 155 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e153) 2019-11-25T15:23:55.494 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:23:55.491+0000 16949700 -1 osd.0 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.783191+0000 front 2019-11-25T15:23:04.786059+0000 (oldest deadline 2019-11-25T15:23:48.062091+0000) 2019-11-25T15:23:55.499 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:23:55.496+0000 16949700 -1 osd.1 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:04.535920+0000 front 2019-11-25T15:23:04.523640+0000 (oldest deadline 2019-11-25T15:23:47.289000+0000) 2019-11-25T15:23:55.622 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:23:55.605+0000 16949700 -1 osd.2 155 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:23:05.040852+0000 front 2019-11-25T15:23:05.323246+0000 (oldest deadline 2019-11-25T15:23:47.865707+0000) 2019-11-25T15:24:07.473 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:09:12.056 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:07.490 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:09:12.074 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:07.499 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:09:12.106 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:07.500 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:09:12.108 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:07.549 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:09:12.157 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:07.551 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:09:12.159 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:07.576 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:09:12.159 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:07.642 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:09:12.249 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:07.642 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:09:12.250 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:24:33.948 INFO:teuthology.orchestra.run.smithi153.stderr:testChunkedPut (test.functional.tests.TestFile) ... ok 2019-11-25T15:24:56.426 INFO:teuthology.orchestra.run.smithi153.stderr:testContentTypeGuessing (test.functional.tests.TestFile) ... ok 2019-11-25T15:25:20.677 INFO:teuthology.orchestra.run.smithi153.stderr:testCopy (test.functional.tests.TestFile) ... ok 2019-11-25T15:25:27.709 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:27.706+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:27.796 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:27.793+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:28.306 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:28.302+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:28.703 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:28.697+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:28.793 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:28.788+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:29.170 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:29.167+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:29.265 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:29.260+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:29.295 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:29.291+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:29.668 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:29.664+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:29.765 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:29.761+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:29.885 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:29.881+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:30.178 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:30.174+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:30.252 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:30.248+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:30.262 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:30.258+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:30.641 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:30.638+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:30.736 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:30.732+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:30.875 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:30.871+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:31.185 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:31.181+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:31.281 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:31.278+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:31.309 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:31.306+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:31.698 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:31.693+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:31.728 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:31.724+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:31.766 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:31.761+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:31.926 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:31.921+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:32.170 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:32.165+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:32.306 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:32.301+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:32.319 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:32.315+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:32.703 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:32.698+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:32.725 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:32.722+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:32.755 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:32.751+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:32.905 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:32.899+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:33.152 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:33.148+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:33.261 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:33.258+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:33.329 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:33.325+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:33.676 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:33.673+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:33.711 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:33.708+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:33.790 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:33.786+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:33.915 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:33.912+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:34.178 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:34.173+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:34.258 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:34.253+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:34.297 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:34.293+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:34.666 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:34.662+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:34.707 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:34.703+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:34.779 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:34.774+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:34.880 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:34.875+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:35.171 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:35.166+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:35.259 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:35.254+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:35.351 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:35.347+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:35.639 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:35.634+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:35.679 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:35.675+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:35.820 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:35.815+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:35.849 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:35.843+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:36.184 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:36.180+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:36.280 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:36.277+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:36.343 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:36.340+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:36.621 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:36.615+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:36.684 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:36.681+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:36.823 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:36.818+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:36.888 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:36.883+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:37.204 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:37.198+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:37.279 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:37.276+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:37.312 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:37.308+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:37.596 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:37.590+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:37.662 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:37.660+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:37.797 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:37.791+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:37.865 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:37.860+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:38.236 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:38.231+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:38.260 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:38.257+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:38.270 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:38.266+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:38.575 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:38.571+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:38.618 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:38.615+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:38.804 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:38.799+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:38.901 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:38.895+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:39.219 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:39.214+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:39.289 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:39.286+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:39.293 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:39.289+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:39.612 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:39.607+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:39.672 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:39.669+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:39.771 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:39.763+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:39.886 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:39.881+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:40.215 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:40.210+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:40.248 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:40.245+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:40.259 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:40.256+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:40.629 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:40.625+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:40.642 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:40.637+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:40.780 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:40.774+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:40.941 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:40.935+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:41.171 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:41.165+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:41.270 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:41.267+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:41.285 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:41.283+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:41.615 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:41.611+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:41.678 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:41.673+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:41.793 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:41.788+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:41.980 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:41.975+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:42.225 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:25:42.220+0000 16949700 -1 osd.5 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.066290+0000 front 2019-11-25T15:24:44.118767+0000 (oldest deadline 2019-11-25T15:25:28.586045+0000) 2019-11-25T15:25:42.249 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:25:42.245+0000 16949700 -1 osd.2 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.002920+0000 front 2019-11-25T15:24:43.574825+0000 (oldest deadline 2019-11-25T15:25:28.849218+0000) 2019-11-25T15:25:42.279 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:42.275+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:25:42.584 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:25:42.580+0000 16949700 -1 osd.0 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:43.883925+0000 front 2019-11-25T15:24:44.016236+0000 (oldest deadline 2019-11-25T15:25:26.752351+0000) 2019-11-25T15:25:42.706 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:25:42.702+0000 16949700 -1 osd.7 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:45.584550+0000 front 2019-11-25T15:24:45.695215+0000 (oldest deadline 2019-11-25T15:25:31.396907+0000) 2019-11-25T15:25:42.821 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:25:42.816+0000 16949700 -1 osd.4 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.922345+0000 front 2019-11-25T15:24:45.567294+0000 (oldest deadline 2019-11-25T15:25:27.195959+0000) 2019-11-25T15:25:42.949 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:25:42.945+0000 16949700 -1 osd.6 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:47.904587+0000 front 2019-11-25T15:24:48.181697+0000 (oldest deadline 2019-11-25T15:25:29.837050+0000) 2019-11-25T15:25:43.318 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:25:43.300+0000 16949700 -1 osd.1 160 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:24:44.575961+0000 front 2019-11-25T15:24:44.569732+0000 (oldest deadline 2019-11-25T15:25:27.368358+0000) 2019-11-25T15:26:12.608 INFO:teuthology.orchestra.run.smithi153.stderr:testCopy404s (test.functional.tests.TestFile) ... ok 2019-11-25T15:26:13.939 INFO:teuthology.orchestra.run.smithi153.stderr:testCopyDestinationSlashProblems (test.functional.tests.TestFile) ... ok 2019-11-25T15:26:38.731 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:38.709+0000 3426e700 -1 osd.4 164 build_incremental_map_msg missing incremental map 164 2019-11-25T15:26:38.749 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:38.738+0000 39278700 -1 osd.7 164 build_incremental_map_msg missing incremental map 164 2019-11-25T15:26:38.865 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:38.854+0000 35270700 -1 osd.4 164 build_incremental_map_msg missing incremental map 164 2019-11-25T15:26:38.908 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:38.857+0000 38a77700 -1 osd.7 164 build_incremental_map_msg missing incremental map 164 2019-11-25T15:26:38.925 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:38.920+0000 3b27c700 -1 osd.4 164 build_incremental_map_msg missing incremental map 164 2019-11-25T15:26:39.135 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:39.112+0000 3aa7b700 -1 osd.4 164 build_incremental_map_msg missing incremental map 164 2019-11-25T15:26:42.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:42.099+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:42.286 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:42.283+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:42.289 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:42.284+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:43.022 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:42.924+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:43.117 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:43.114+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:43.257 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:43.253+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:43.310 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:43.305+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:43.520 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:11:48.104 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.523 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:11:48.106 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.569 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:11:48.176 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.571 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:11:48.179 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.584 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:11:48.191 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.592 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:11:48.200 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.694 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:11:48.278 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.742 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:11:48.350 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.779 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:11:48.386 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:43.932 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:43.927+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:43.970 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:43.967+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:44.009 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:44.005+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:44.113 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:44.111+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:44.297 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:44.291+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:44.304 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:44.298+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:44.918 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:44.914+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:44.961 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:44.957+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:44.963 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:44.961+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:45.074 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:45.072+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:45.289 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:45.286+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:45.335 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:45.330+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:45.878 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:45.874+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:45.946 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:45.944+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:45.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:45.973+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:46.117 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:46.115+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:46.288 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:46.286+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:46.402 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:46.392+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:46.912 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:46.907+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:46.974 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:46.971+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:47.026 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:47.021+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:47.156 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:47.154+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:47.329 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:47.326+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:47.413 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:47.409+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:47.940 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:47.936+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:48.004 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:48.000+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:48.007 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:48.004+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:48.142 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:48.139+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:48.345 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:48.343+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:48.453 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:48.449+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:48.943 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:48.939+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:48.961 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:48.959+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:49.033 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:49.029+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:49.163 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:49.158+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:49.316 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:49.312+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:49.428 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:49.424+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:49.926 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:49.921+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:49.959 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:49.956+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:50.048 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:50.043+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:50.160 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:50.158+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:50.308 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:50.304+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:50.429 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:50.423+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:50.911 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:50.905+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:50.984 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:50.981+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:51.014 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:51.009+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:51.136 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:51.132+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:51.271 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:51.269+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:51.475 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:51.470+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:51.914 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:51.909+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:52.019 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:52.016+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:52.061 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:52.057+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:52.109 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:52.106+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:52.233 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:52.230+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:52.503 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:52.497+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:52.902 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:26:52.898+0000 16949700 -1 osd.5 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:26:00.979721+0000 front 2019-11-25T15:26:01.139373+0000 (oldest deadline 2019-11-25T15:26:43.804161+0000) 2019-11-25T15:26:52.973 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:26:52.970+0000 16949700 -1 osd.1 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.598631+0000 front 2019-11-25T15:25:58.621791+0000 (oldest deadline 2019-11-25T15:26:42.691718+0000) 2019-11-25T15:26:53.111 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:26:53.108+0000 16949700 -1 osd.0 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.590024+0000 front 2019-11-25T15:25:58.198849+0000 (oldest deadline 2019-11-25T15:26:41.242058+0000) 2019-11-25T15:26:53.114 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:26:53.109+0000 16949700 -1 osd.7 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:59.958137+0000 front 2019-11-25T15:25:58.870423+0000 (oldest deadline 2019-11-25T15:26:43.866438+0000) 2019-11-25T15:26:53.225 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:26:53.223+0000 16949700 -1 osd.2 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:58.591607+0000 front 2019-11-25T15:25:58.404733+0000 (oldest deadline 2019-11-25T15:26:41.772049+0000) 2019-11-25T15:26:53.553 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:53.548+0000 36a73700 -1 osd.4 166 build_incremental_map_msg missing incremental map 164 2019-11-25T15:26:53.561 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:26:53.558+0000 16949700 -1 osd.4 166 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:25:55.586040+0000 front 2019-11-25T15:25:55.623363+0000 (oldest deadline 2019-11-25T15:26:41.434934+0000) 2019-11-25T15:26:53.678 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:11:58.262 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:53.690 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:11:58.274 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:53.692 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:11:58.299 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:53.700 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:11:58.308 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:53.702 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:11:58.309 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:53.707 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:11:58.314 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:53.735 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:11:58.319 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:53.740 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:11:58.348 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:26:53.742 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:11:58.349 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:27:13.802 INFO:teuthology.orchestra.run.smithi153.stderr:testCopyFromHeader (test.functional.tests.TestFile) ... ok 2019-11-25T15:27:27.897 INFO:teuthology.orchestra.run.smithi153.stderr:testCopyFromHeader404s (test.functional.tests.TestFile) ... ok 2019-11-25T15:27:30.047 INFO:teuthology.orchestra.run.smithi153.stderr:testCopyNoDestinationHeader (test.functional.tests.TestFile) ... ok 2019-11-25T15:27:36.371 INFO:teuthology.orchestra.run.smithi153.stderr:testDelete (test.functional.tests.TestFile) ... ok 2019-11-25T15:27:37.148 INFO:teuthology.orchestra.run.smithi153.stderr:testDeleteOfFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-11-25T15:27:42.893 INFO:teuthology.orchestra.run.smithi153.stderr:testDeleteThen404s (test.functional.tests.TestFile) ... ok 2019-11-25T15:27:44.162 INFO:teuthology.orchestra.run.smithi153.stderr:testEtagResponse (test.functional.tests.TestFile) ... ok 2019-11-25T15:27:44.270 INFO:teuthology.orchestra.run.smithi153.stderr:testEtagWayoff (test.functional.tests.TestFile) ... ok 2019-11-25T15:27:58.814 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:27:58.796+0000 39278700 -1 osd.7 170 build_incremental_map_msg missing incremental map 164 2019-11-25T15:28:02.480 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:28:02.475+0000 3a27a700 -1 osd.6 172 build_incremental_map_msg missing incremental map 172 2019-11-25T15:28:02.552 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:28:02.550+0000 3b27c700 -1 osd.6 172 build_incremental_map_msg missing incremental map 172 2019-11-25T15:28:10.813 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:28:10.798+0000 35a71700 -1 osd.6 173 build_incremental_map_msg missing incremental map 172 2019-11-25T15:28:48.135 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:48.130+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:49.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:49.099+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:50.094 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:50.091+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:51.116 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:51.113+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:52.097 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:52.094+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:53.145 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:53.143+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:54.110 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:54.106+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:55.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:55.063+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:56.053 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:56.050+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:57.045 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:57.041+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:58.023 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:58.020+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:58.984 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:58.980+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:28:59.961 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:28:59.957+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:01.002 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:01.000+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:01.972 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:01.970+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:03.003 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:03.001+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:04.047 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:04.044+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:04.910 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:04.904+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:04.986 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:04.981+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:05.063 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:05.060+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:05.871 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:05.867+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:06.005 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:06.000+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:06.085 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:06.076+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:06.107 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:06.104+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:06.108 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:06.106+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:06.315 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:06.308+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:06.832 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:06.829+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:06.906 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:06.901+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:06.990 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:06.985+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:07.061 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:07.055+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:07.068 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:07.066+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:07.070 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:07.068+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:07.365 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:07.360+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:07.728 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:07.726+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:07.820 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:07.816+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:07.916 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:07.906+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:08.040 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:08.035+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:08.043 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:08.039+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:08.043 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:08.041+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:08.076 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:08.070+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:08.332 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:08.327+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:08.707 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:08.705+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:08.829 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:08.824+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:08.907 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:08.902+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:08.999 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:08.996+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:09.000 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:08.999+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:09.033 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:09.028+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:09.123 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:09.118+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:09.382 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:09.377+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:09.760 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:09.757+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:09.808 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:09.804+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:09.878 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:09.872+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:09.968 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:09.965+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:09.969 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:09.966+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:10.016 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:10.010+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:10.087 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:10.082+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:10.346 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:10.340+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:10.761 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:10.755+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:10.807 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:10.804+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:10.902 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:10.896+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:10.933 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:10.924+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:10.934 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:10.927+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:10.996 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:10.991+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:11.118 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:11.114+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:11.316 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:11.311+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:11.713 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:11.709+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:11.840 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:11.834+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:11.947 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:11.939+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:11.947 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:11.941+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:11.951 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:11.946+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:11.980 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:11.974+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:12.143 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:12.137+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:12.287 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:12.281+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:12.700 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:12.695+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:12.876 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:12.872+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:12.963 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:12.958+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:12.994 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:12.990+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:12.996 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:12.992+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:13.005 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:12.999+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:13.194 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:13.189+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:13.303 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:13.298+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:13.669 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:13.665+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:13.884 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:13.880+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:13.958 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:13.954+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:13.959 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:13.956+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:13.986 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:13.982+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:14.024 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:14.008+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:14.165 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:14.159+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:14.336 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:14.331+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:14.655 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:14.652+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:14.925 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:14.923+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:14.948 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:14.945+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:14.948 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:14.946+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:14.985 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:14.969+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:14.991 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:14.981+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:15.163 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:15.158+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:15.342 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:15.336+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:15.705 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:15.701+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:15.942 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:15.938+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:15.943 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:15.940+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:15.965 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:15.952+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:16.011 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:16.005+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:16.027 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:16.021+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:16.138 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:16.132+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:16.308 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:16.303+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:16.670 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:16.665+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:16.949 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:16.945+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:16.950 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:16.947+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:16.968 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:16.962+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:16.974 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:16.971+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:17.057 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:17.051+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:17.096 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:17.090+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:17.264 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:17.258+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:17.710 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:17.706+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:17.919 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:17.917+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:17.921 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:17.919+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:17.970 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:17.967+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:18.015 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:18.010+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:18.104 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:18.098+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:18.108 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:18.103+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:18.304 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:18.299+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:18.733 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:18.730+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:18.912 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:18.908+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:18.913 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:18.910+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:19.002 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:18.997+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:19.028 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:19.022+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:19.142 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:19.137+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:19.148 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:19.144+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:19.272 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:19.266+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:19.749 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:19.746+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:19.949 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:19.946+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:19.951 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:19.949+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:20.040 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:20.036+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:20.050 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:20.044+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:20.115 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:20.110+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:20.132 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:20.127+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:20.256 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:20.250+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:20.743 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:20.738+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:20.956 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:20.951+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:20.956 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:20.954+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:21.001 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:20.997+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:21.092 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:21.086+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:21.101 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:21.097+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:21.106 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:21.101+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:21.242 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:21.237+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:21.745 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:21.742+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:21.959 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:21.954+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:21.982 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:21.977+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:21.983 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:21.980+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:22.068 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:22.064+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:22.119 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:22.116+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:22.143 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:22.140+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:22.251 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:22.246+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:22.758 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:22.754+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:22.971 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:22.968+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:23.025 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:23.022+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:23.026 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:23.024+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:23.027 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:23.022+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:23.094 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:23.089+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:23.165 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:23.160+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:23.288 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:23.283+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:23.764 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:23.760+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:23.972 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:23.968+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:24.048 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:24.043+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:24.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:24.062+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:24.067 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:24.064+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:24.091 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:24.086+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:24.214 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:24.209+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:24.335 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:24.330+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:24.815 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:24.813+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:24.970 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:24.966+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:25.061 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:25.055+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:25.067 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:25.063+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:25.067 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:25.065+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:25.092 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:25.086+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:25.269 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:25.264+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:25.325 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:25.320+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:25.847 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:25.843+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:25.957 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:25.953+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:26.054 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:26.047+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:26.067 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:26.059+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:26.096 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:26.092+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:26.097 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:26.093+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:26.284 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:26.278+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:26.287 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:26.283+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:26.889 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:26.886+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:26.997 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:26.993+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:27.037 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:27.032+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:27.046 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:27.041+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:27.146 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:27.142+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:27.147 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:27.144+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:27.302 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:27.296+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:27.313 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:27.308+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:27.869 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:27.865+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:27.981 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:27.977+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:28.027 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:28.021+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:28.059 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:28.053+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:28.137 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:28.133+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:28.137 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:28.135+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:28.325 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:28.320+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:28.354 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:28.349+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:28.898 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:28.895+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:28.936 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:28.932+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:29.023 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:29.015+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:29.092 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:29.087+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:29.121 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:29.117+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:29.124 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:29.121+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:29.309 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:29.304+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:29.320 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:29.316+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:29.957 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:29.943+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:29.965 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:29.961+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:29.980 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:29.975+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:30.099 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:30.096+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:30.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:30.099+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:30.108 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:30.103+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:30.333 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:30.328+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:30.357 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:30.351+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:30.963 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:30.959+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:30.981 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:30.978+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:31.001 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:30.996+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:31.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:31.097+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:31.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:31.099+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:31.113 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:31.108+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:31.341 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:31.336+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:31.341 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:31.335+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:31.942 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:31.938+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:31.986 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:31.984+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:32.035 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:32.029+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:32.069 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:32.065+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:32.069 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:32.067+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:32.087 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:32.082+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:32.365 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:32.355+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:32.376 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:32.371+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:32.982 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:32.978+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:33.023 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:33.019+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:33.038 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:33.035+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:33.039 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:33.038+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:33.060 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:33.055+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:33.070 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:33.065+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:33.339 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:33.333+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:33.384 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:29:33.379+0000 16949700 -1 osd.6 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:21.377720+0000 front 2019-11-25T15:28:21.379841+0000 (oldest deadline 2019-11-25T15:29:05.691499+0000) 2019-11-25T15:29:34.031 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:29:34.026+0000 16949700 -1 osd.1 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.219842+0000 front 2019-11-25T15:28:22.224988+0000 (oldest deadline 2019-11-25T15:29:07.508195+0000) 2019-11-25T15:29:34.055 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:34.052+0000 16949700 -1 osd.0 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:20.563864+0000 front 2019-11-25T15:28:20.717302+0000 (oldest deadline 2019-11-25T15:29:05.773291+0000) 2019-11-25T15:29:34.056 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:34.053+0000 16949700 -1 osd.0 173 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:34.062 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:29:34.058+0000 16949700 -1 osd.2 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:22.726054+0000 front 2019-11-25T15:28:22.558470+0000 (oldest deadline 2019-11-25T15:29:06.555590+0000) 2019-11-25T15:29:34.065 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:29:34.060+0000 16949700 -1 osd.4 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.313446+0000 front 2019-11-25T15:28:18.784212+0000 (oldest deadline 2019-11-25T15:29:04.127768+0000) 2019-11-25T15:29:34.111 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:29:34.106+0000 16949700 -1 osd.7 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:19.341562+0000 front 2019-11-25T15:28:19.580767+0000 (oldest deadline 2019-11-25T15:29:04.593725+0000) 2019-11-25T15:29:34.336 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:29:34.332+0000 16949700 -1 osd.5 173 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:28:23.952170+0000 front 2019-11-25T15:28:23.976570+0000 (oldest deadline 2019-11-25T15:29:05.642925+0000) 2019-11-25T15:29:35.029 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:35.026+0000 16949700 -1 osd.0 174 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:36.087 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:36.076+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:37.067 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:37.064+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:38.036 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:38.034+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:39.079 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:39.077+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:40.076 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:40.074+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:41.048 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:41.045+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:42.069 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:42.067+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:43.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:43.063+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:44.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:44.064+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:45.026 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:45.023+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:45.990 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:45.987+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:47.007 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:47.003+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:47.966 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:47.963+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:48.961 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:48.955+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:49.963 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:49.960+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:50.994 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:29:50.991+0000 16949700 -1 osd.0 175 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:109779 2.26s0 2:6471313f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_8ac71497bfaf414b9abced01d3845fcf:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~17211 in=17211b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e173) 2019-11-25T15:29:55.581 INFO:teuthology.orchestra.run.smithi153.stderr:testFileCreate (test.functional.tests.TestFile) ... ok 2019-11-25T15:29:56.988 INFO:teuthology.orchestra.run.smithi153.stderr:testGetContentType (test.functional.tests.TestFile) ... ok 2019-11-25T15:30:02.610 INFO:teuthology.orchestra.run.smithi153.stderr:testGetOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-11-25T15:30:04.343 INFO:teuthology.orchestra.run.smithi153.stderr:testHead (test.functional.tests.TestFile) ... ok 2019-11-25T15:30:04.972 INFO:teuthology.orchestra.run.smithi153.stderr:testHeadOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-11-25T15:30:29.203 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:29.198+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:29.646 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:29.643+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:30.209 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:30.206+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:30.424 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:30.420+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:30.541 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:30.537+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:30.638 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:30.635+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:31.186 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:31.182+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:31.440 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:31.436+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:31.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:31.471+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:31.583 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:31.580+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:31.596 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:31.592+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:32.147 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:32.144+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:32.338 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:32.336+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:32.454 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:32.451+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:32.475 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:32.472+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:32.560 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:32.556+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:32.563 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:32.560+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:33.193 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:33.190+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:33.350 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:33.346+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:33.448 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:33.445+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:33.470 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:33.466+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:33.549 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:33.546+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:33.572 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:33.568+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:34.162 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:34.158+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:34.328 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:34.324+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:34.473 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:34.468+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:34.492 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:34.487+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:34.505 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:34.502+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:34.576 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:34.573+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:35.202 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:35.199+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:35.328 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:35.325+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:35.476 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:35.472+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:35.484 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:35.480+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:35.493 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:35.490+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:35.587 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:35.583+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:36.182 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:36.176+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:36.369 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:36.366+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:36.467 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:36.463+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:36.483 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:36.480+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:36.496 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:36.493+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:36.598 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:36.594+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:37.215 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:37.211+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:37.389 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:37.386+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:37.478 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:37.474+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:37.499 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:37.495+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:37.503 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:37.500+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:37.593 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:37.588+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:38.239 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:38.234+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:38.407 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:38.404+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:38.454 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:38.449+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:38.504 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:38.501+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:38.514 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:38.512+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:38.639 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:38.634+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:39.212 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:39.208+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:39.380 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:39.376+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:39.449 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:39.444+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:39.526 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:39.522+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:39.534 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:39.530+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:39.644 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:15:44.228 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:39.656 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:39.652+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:39.660 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:15:44.244 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:39.674 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:15:44.282 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:39.685 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:15:44.292 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:39.689 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:15:44.296 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:39.697 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:15:44.304 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:39.936 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:15:44.544 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:39.963 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:15:44.547 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:39.991 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:15:44.599 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:30:40.190 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:40.185+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:40.356 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:40.353+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:40.494 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:40.487+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:40.498 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:40.494+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:40.528 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:40.524+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:40.678 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:40.675+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:41.227 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:41.223+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:41.405 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:41.401+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:41.497 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:41.493+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:41.508 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:41.504+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:41.567 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:41.564+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:41.644 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:41.640+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:42.202 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:42.195+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:42.363 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:42.359+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:42.495 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:42.490+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:42.553 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:42.548+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:42.616 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:42.612+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:42.638 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:42.632+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:43.181 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:43.176+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:43.374 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:43.370+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:43.544 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:43.540+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:43.591 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:43.586+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:43.617 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:43.612+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:43.682 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:43.678+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:44.160 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:44.156+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:44.341 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:44.336+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:44.575 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:44.571+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:44.591 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:44.587+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:44.609 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:44.606+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:44.644 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:44.640+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:45.160 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:45.155+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:45.346 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:45.343+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:45.571 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:45.566+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:45.581 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:45.579+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:45.628 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:45.624+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:45.655 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:45.652+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:46.209 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:46.204+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:46.377 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:46.373+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:46.545 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:46.541+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:46.609 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:46.606+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:46.627 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:46.624+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:46.709 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:46.705+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:47.178 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:47.173+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:47.389 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:47.386+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:47.546 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:47.541+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:47.594 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:47.590+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:47.621 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:47.617+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:47.687 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:47.684+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:48.136 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:48.131+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:48.409 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:48.405+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:48.552 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:48.548+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:48.555 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:48.551+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:48.594 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:48.591+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:48.735 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:48.731+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:49.190 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:49.185+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:49.239 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:49.237+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:49.384 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:49.382+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:49.520 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:49.515+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:49.542 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:49.538+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:49.589 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:49.585+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:49.756 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:49.753+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:50.183 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:50.178+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:50.246 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:50.244+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:50.393 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:50.390+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:50.536 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:50.531+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:50.560 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:50.556+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:50.573 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:50.570+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:50.778 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:50.774+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:51.185 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:51.180+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:51.259 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:51.256+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:51.410 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:51.407+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:51.557 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:51.553+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:51.565 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:51.562+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:51.582 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:51.578+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:51.779 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:51.776+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:52.204 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:52.200+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:52.305 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:52.303+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:52.439 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:52.437+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:52.523 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:52.519+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:52.596 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:52.591+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:52.597 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:52.594+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:52.768 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:52.764+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:53.180 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:53.175+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:53.287 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:53.285+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:53.455 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:53.451+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:53.501 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:53.496+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:53.569 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:53.564+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:53.578 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:53.574+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:53.746 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:53.743+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:54.148 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:54.143+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:54.336 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:54.334+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:54.496 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:54.493+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:54.527 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:54.523+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:54.531 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:54.527+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:54.614 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:54.612+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:54.754 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:54.751+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:55.171 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:55.167+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:55.360 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:55.351+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:30:55.361 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:55.358+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:55.456 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:55.453+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:55.517 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:55.508+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:55.533 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:55.529+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:55.655 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:55.653+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:55.789 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:55.786+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:56.129 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:56.124+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:56.373 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:56.370+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:30:56.376 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:56.374+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:56.442 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:56.440+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:56.541 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:56.531+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:56.574 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:56.570+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:56.682 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:56.679+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:56.806 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:56.802+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:57.093 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:57.089+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:57.424 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:57.420+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:30:57.425 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:57.422+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:57.459 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:57.453+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:57.583 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:57.579+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:57.625 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:57.621+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:57.725 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:57.721+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:57.849 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:57.845+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:58.084 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:58.080+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:58.394 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:58.389+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:30:58.394 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:58.391+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:58.446 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:58.442+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:58.546 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:58.540+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:58.593 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:58.589+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:58.742 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:58.739+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:58.871 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:58.868+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:30:59.075 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:30:59.070+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:30:59.429 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:59.426+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:30:59.430 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:30:59.427+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:30:59.431 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:30:59.427+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:30:59.546 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:30:59.542+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:30:59.608 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:30:59.603+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:30:59.793 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:30:59.789+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:30:59.916 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:30:59.912+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:00.083 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:00.078+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:31:00.415 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:00.411+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:00.455 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:00.451+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:00.456 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:00.453+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:00.512 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:00.507+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:00.578 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:00.574+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:00.793 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:00.790+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:00.914 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:00.912+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:01.038 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:01.033+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:31:01.424 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:01.421+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:01.426 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:01.424+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:01.459 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:01.455+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:01.526 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:01.522+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:01.578 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:01.574+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:01.793 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:01.790+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:01.902 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:01.898+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:02.083 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:02.079+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:31:02.396 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:02.392+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:02.397 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:02.394+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:02.438 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:02.435+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:02.484 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:02.479+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:02.603 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:02.598+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:02.752 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:02.746+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:02.897 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:02.894+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:03.055 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:03.050+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:31:03.361 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:03.357+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:03.362 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:03.358+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:03.402 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:03.398+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:03.501 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:03.495+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:03.624 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:03.553+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:03.739 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:03.734+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:03.904 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:03.900+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:04.041 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:04.036+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:31:04.324 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:04.321+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:04.324 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:04.322+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:04.421 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:04.416+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:04.459 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:04.455+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:04.523 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:31:04.496+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 48sec e178 v178) 2019-11-25T15:31:04.602 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:04.597+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:04.693 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:04.690+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:04.933 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:04.929+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:05.069 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:05.051+0000 16949700 -1 osd.4 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:42.670180+0000 front 2019-11-25T15:29:52.716383+0000 (oldest deadline 2019-11-25T15:30:28.328658+0000) 2019-11-25T15:31:05.069 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:05.062+0000 16949700 -1 osd.4 178 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:05.330 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:05.327+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:05.333 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:05.331+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:05.438 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:05.432+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:05.507 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:05.503+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:05.589 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:05.585+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:05.676 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:05.673+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:05.873 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T15:31:05.639+0000 20e21700 -1 RGWWatcher::handle_error cookie 821398976 err (107) Transport endpoint is not connected 2019-11-25T15:31:05.945 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:05.932+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:06.108 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:06.104+0000 16949700 -1 osd.4 178 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:06.375 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:06.372+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:06.378 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:06.375+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:06.481 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:06.478+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:06.527 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:06.523+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:06.623 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:06.619+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:06.651 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:06.649+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:06.915 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:06.912+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:07.140 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:07.136+0000 16949700 -1 osd.4 178 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:07.429 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:07.426+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:07.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:07.429+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:07.507 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:07.503+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:07.570 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:07.565+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:07.617 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:07.613+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:07.654 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:07.651+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:07.929 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:07.926+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:08.156 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:08.152+0000 16949700 -1 osd.4 178 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:08.425 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:08.421+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:08.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:08.425+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:08.491 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:08.488+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:08.559 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:08.554+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:08.617 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:08.612+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:08.627 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:08.624+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:08.958 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:08.954+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:09.167 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:09.163+0000 16949700 -1 osd.4 178 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:09.446 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:09.443+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:09.450 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:09.448+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:09.483 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:09.480+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:09.524 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:31:09.518+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 48sec e178 v178) 2019-11-25T15:31:09.578 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:09.574+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:09.594 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:09.589+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:09.642 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:09.640+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:09.921 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:09.915+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:10.133 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:10.129+0000 16949700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:10.417 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:10.413+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:10.418 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:10.415+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:10.464 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:10.461+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:10.562 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:10.557+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:10.574 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:10.570+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:10.660 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:10.657+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:10.876 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:10.871+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:11.123 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:11.115+0000 16949700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:11.372 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:11.369+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:11.375 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:11.372+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:11.472 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:11.469+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:11.576 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:11.572+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:11.608 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:11.603+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:11.622 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:11.619+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:11.929 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:11.925+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:12.101 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:12.097+0000 16949700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:12.379 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:12.376+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:12.382 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:12.380+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:12.449 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:12.445+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:12.564 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:12.560+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:12.583 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:12.579+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:12.594 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:12.592+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:12.901 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:12.897+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:13.130 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:13.126+0000 16949700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:13.376 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:13.373+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:13.378 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:13.374+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:13.406 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:13.403+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:13.557 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:13.551+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:13.557 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:13.553+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:13.572 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:13.568+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:13.879 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:13.876+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:14.129 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:14.125+0000 16949700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:14.337 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:14.334+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:14.339 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:14.336+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:14.459 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:14.456+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:14.543 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:31:14.537+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 48sec e178 v178) 2019-11-25T15:31:14.548 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:14.545+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:14.584 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:14.580+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:14.610 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:14.605+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:14.846 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:14.842+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:15.174 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:15.171+0000 16949700 -1 osd.4 178 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:15.309 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:15.306+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:15.309 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:15.307+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:15.424 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:15.421+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:15.528 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:15.525+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:15.546 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:15.542+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:15.627 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:15.624+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:15.894 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:15.892+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:16.149 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:16.146+0000 16949700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:16.306 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:16.302+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:16.306 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:16.303+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:16.403 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:16.400+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:16.536 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:16.533+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:16.587 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:16.584+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:16.588 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:16.585+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:16.891 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:16.887+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:17.152 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:17.148+0000 16949700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:17.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:17.350+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:17.357 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:17.354+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:17.454 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:17.450+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:17.558 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:17.555+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:17.569 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:17.566+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:17.570 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:17.567+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:17.899 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:17.896+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:18.187 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:18.184+0000 16949700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:18.364 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:18.361+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:18.364 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:18.362+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:18.423 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:18.419+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:18.522 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:18.518+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:18.568 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:18.565+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:18.570 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:18.567+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:18.901 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:18.899+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:19.143 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:19.140+0000 16949700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:19.402 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:19.399+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:19.403 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:19.399+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:19.403 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:19.401+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:19.522 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:19.518+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:19.564 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:31:19.557+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 48sec e178 v178) 2019-11-25T15:31:19.573 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:19.570+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:19.614 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:19.610+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:19.898 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:19.894+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:20.112 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:20.109+0000 16949700 -1 osd.4 178 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:20.385 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:20.381+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:20.436 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:20.432+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:20.437 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:20.434+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:20.511 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:20.508+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:20.573 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:20.569+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:20.600 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:20.597+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:20.948 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:20.946+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:21.160 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:21.157+0000 16949700 -1 osd.4 178 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:21.432 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:21.428+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:21.484 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:21.480+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:21.484 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:21.482+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:21.498 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:21.495+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:21.600 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:21.597+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:21.650 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:21.646+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:21.952 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:21.949+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:22.193 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:22.189+0000 16949700 -1 osd.4 178 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:22.429 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:22.425+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:22.482 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:22.478+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:22.483 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:22.480+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:22.488 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:22.486+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:22.591 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:22.588+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:22.652 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:22.649+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:22.965 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:22.962+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:23.208 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:23.205+0000 16949700 -1 osd.4 178 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:23.465 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:23.462+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:23.490 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:23.488+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:23.512 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:23.508+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:23.513 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:23.510+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:23.618 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:23.615+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:23.683 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:23.680+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:23.982 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:23.978+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:24.175 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:24.173+0000 16949700 -1 osd.4 178 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:24.439 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:31:24.436+0000 16949700 -1 osd.2 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.054942+0000 front 2019-11-25T15:29:52.654826+0000 (oldest deadline 2019-11-25T15:30:32.229500+0000) 2019-11-25T15:31:24.475 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:31:24.472+0000 16949700 -1 osd.1 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:51.674007+0000 front 2019-11-25T15:29:51.715208+0000 (oldest deadline 2019-11-25T15:30:28.821664+0000) 2019-11-25T15:31:24.566 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:24.561+0000 16949700 -1 osd.3 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:30:14.417715+0000 (oldest deadline 2019-11-25T15:30:54.417715+0000) 2019-11-25T15:31:24.571 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:31:24.567+0000 16949700 -1 osd.3 178 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e177) 2019-11-25T15:31:24.642 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:31:24.639+0000 16949700 -1 osd.7 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.193206+0000 front 2019-11-25T15:29:52.117872+0000 (oldest deadline 2019-11-25T15:30:29.568659+0000) 2019-11-25T15:31:24.652 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:31:24.648+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 48sec e178 v178) 2019-11-25T15:31:24.679 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:31:24.677+0000 16949700 -1 osd.5 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:52.642669+0000 front 2019-11-25T15:29:51.823559+0000 (oldest deadline 2019-11-25T15:30:31.340396+0000) 2019-11-25T15:31:24.946 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:31:24.943+0000 16949700 -1 osd.0 178 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:29:48.343583+0000 front 2019-11-25T15:29:52.199861+0000 (oldest deadline 2019-11-25T15:30:30.244081+0000) 2019-11-25T15:31:25.411 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:25.408+0000 16949700 -1 osd.4 179 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:26.431 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:26.419+0000 16949700 -1 osd.4 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:27.420 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:27.417+0000 16949700 -1 osd.4 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:28.460 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:28.456+0000 16949700 -1 osd.4 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:29.424 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:29.418+0000 16949700 -1 osd.4 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:30.439 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:30.435+0000 16949700 -1 osd.4 180 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:31.467 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:31.463+0000 16949700 -1 osd.4 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:32.485 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:32.477+0000 16949700 -1 osd.4 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:33.532 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:33.528+0000 16949700 -1 osd.4 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:34.502 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:34.498+0000 16949700 -1 osd.4 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:35.508 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:35.504+0000 16949700 -1 osd.4 180 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:36.562 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:36.557+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:37.546 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:37.541+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:38.586 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:38.581+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:39.639 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:39.635+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:40.651 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:40.646+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:41.681 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:41.675+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:42.686 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:42.683+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:43.657 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:43.653+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:44.635 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:44.630+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:45.629 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:31:45.625+0000 16949700 -1 osd.4 180 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:110058 5.3 5:c609908c:::notify.5:head [watch ping cookie 821398976 gen 77] snapc 0=[] ondisk+write+known_if_redirected e178) 2019-11-25T15:31:50.576 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.573+0000 37a75700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.638 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.624+0000 38276700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.677 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.643+0000 36272700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.749 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.650+0000 3a27a700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.850 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.778+0000 36a73700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.874 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.824+0000 3b27c700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.878 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.834+0000 38276700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.882 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.847+0000 39a79700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.891 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.850+0000 38276700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.907 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.879+0000 38a77700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.920 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.885+0000 39278700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:31:50.946 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:31:50.920+0000 35270700 -1 osd.6 183 build_incremental_map_msg missing incremental map 183 2019-11-25T15:32:22.381 INFO:teuthology.orchestra.run.smithi153.stderr:testMetadataOnPost (test.functional.tests.TestFile) ... ok 2019-11-25T15:32:30.196 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:17:34.780 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:32:30.206 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:17:34.790 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:32:30.225 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:17:34.833 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:32:30.237 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:17:34.845 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:32:30.242 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:17:34.850 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:32:30.243 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:17:34.851 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:32:30.344 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:17:34.952 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:32:30.355 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:17:34.939 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:32:30.408 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:17:35.016 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:33:10.017 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:10.012+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:11.004 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:11.000+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:11.238 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:11.228+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:11.248 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:11.244+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:11.551 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:11.548+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:11.842 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:11.838+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:11.982 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:11.977+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:12.249 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:12.243+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:12.266 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:12.262+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:12.274 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:12.269+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:12.590 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:12.587+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:12.822 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:12.819+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:13.022 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:13.016+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:13.269 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:13.264+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:13.305 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:13.298+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:13.314 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:13.310+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:13.622 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:13.617+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:13.876 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:13.871+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:14.069 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:14.064+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:14.288 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:14.281+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:14.294 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:14.290+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:14.354 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:14.350+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:14.588 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:14.584+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:14.809 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:14.804+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:14.881 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:14.875+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:15.041 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:15.036+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:15.310 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:15.306+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:15.313 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:15.308+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:15.379 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:15.376+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:15.572 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:15.567+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:15.832 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:15.818+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:15.865 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:15.856+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:16.024 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:16.018+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:16.313 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:16.309+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:16.342 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:16.339+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:16.345 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:16.340+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:16.621 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:16.617+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:16.814 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:16.808+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:16.821 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:16.817+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:17.043 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:17.037+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:17.273 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:17.267+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:17.360 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:17.354+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:17.391 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:17.386+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:17.592 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:17.588+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:17.779 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:17.771+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:17.839 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:17.836+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:18.094 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:18.088+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:18.240 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:18.235+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:18.399 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:18.394+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:18.405 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:18.400+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:18.605 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:18.598+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:18.756 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:18.750+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:18.871 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:18.866+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:19.057 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:19.051+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:19.203 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:19.198+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:19.384 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:19.378+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:19.385 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:19.380+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:19.568 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:19.563+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:19.758 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:19.752+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:19.869 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:19.865+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:20.034 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:20.028+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:20.173 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:20.167+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:20.350 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:20.346+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:20.361 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:20.356+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:20.620 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:20.614+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:20.781 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:20.776+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:20.828 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:20.823+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:21.075 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:21.068+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:21.139 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:21.134+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:21.341 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:21.336+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:21.411 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:21.407+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:21.628 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:21.624+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:21.834 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:21.829+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:21.859 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:21.856+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:22.115 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:22.109+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:22.172 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:22.167+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:22.304 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:22.298+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:22.370 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:22.366+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:22.608 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:22.604+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:22.796 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:22.791+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:22.895 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:22.892+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:23.157 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:23.153+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:23.176 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:23.171+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:23.284 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:23.278+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:23.400 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:23.397+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:23.595 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:23.591+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:23.771 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:23.767+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:23.875 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:23.870+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:24.149 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:24.141+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:24.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:24.220+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:24.311 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:24.306+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:24.404 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:24.400+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:24.577 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:24.574+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:24.826 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:24.820+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:24.858 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:24.853+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:25.133 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:25.129+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:25.226 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:25.220+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:25.276 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:25.270+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:25.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:25.444+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:25.597 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:25.594+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:25.862 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:25.856+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:25.872 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:25.867+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:26.112 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:26.107+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:26.256 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:26.251+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:26.313 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:26.307+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:26.423 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:26.420+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:26.557 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:26.550+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:26.846 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:26.842+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:26.887 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:26.883+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:27.099 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:27.093+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:27.263 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:27.258+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:27.316 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:27.312+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:27.435 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:27.431+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:27.597 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:27.594+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:27.823 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:27.820+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:27.861 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:27.856+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:28.132 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:28.127+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:28.267 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:28.261+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:28.322 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:28.316+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:28.419 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:28.415+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:28.628 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:28.623+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:28.824 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:28.819+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:28.871 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:28.867+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:29.108 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:29.097+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:29.237 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:29.232+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:29.309 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:29.300+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:29.436 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:29.432+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:29.659 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:29.656+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:29.867 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:29.862+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:29.872 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:29.868+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:30.131 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:30.126+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:30.241 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:30.236+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:30.297 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:30.291+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:30.419 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:30.416+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:30.644 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:30.641+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:30.872 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:30.867+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:30.923 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:30.919+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:31.140 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:31.134+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:31.258 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:31.253+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:31.299 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:31.294+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:31.437 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:31.433+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:31.695 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:31.691+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:31.848 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:31.843+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:31.943 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:31.940+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:32.129 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:32.124+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:32.266 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:32.260+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:32.279 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:32.274+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:32.482 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:32.479+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:32.651 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:32.648+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:32.876 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:32.871+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:32.924 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:32.919+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:33.158 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:33.152+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:33.234 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:33.229+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:33.277 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:33.271+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:33.503 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:33.499+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:33.624 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:33.621+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:33.887 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:33.884+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:33.894 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:33.889+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:34.191 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:34.186+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:34.283 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:34.279+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:34.306 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:34.301+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:34.486 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:34.475+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:34.628 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:34.624+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:34.877 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:34.869+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:34.879 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:34.874+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:35.192 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:35.185+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:35.288 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:35.283+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:35.299 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:35.293+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:35.498 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:35.494+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:35.642 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:35.638+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:35.842 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:35.838+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:35.899 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:35.894+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:36.156 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:36.151+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:36.261 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:36.255+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:36.298 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:36.292+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:36.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:36.472+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:36.613 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:36.609+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:36.840 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:36.837+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:36.871 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:36.866+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:37.112 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:37.107+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:37.251 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:37.246+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:37.295 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:37.290+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:37.453 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:37.450+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:37.641 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:37.638+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:37.850 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:37.845+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:37.887 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:37.882+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:38.156 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:38.144+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:38.305 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:38.299+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:38.333 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:38.328+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:38.504 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:38.501+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:38.688 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:38.683+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:38.830 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:38.826+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:38.934 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:38.930+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:39.160 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:39.154+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:39.276 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:39.270+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:39.390 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:39.381+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:39.513 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:39.509+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:39.715 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:39.709+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:39.846 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:39.844+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:39.924 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:39.918+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:40.214 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:40.208+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:40.249 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:40.244+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:40.341 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:33:40.338+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 44sec e183 v183) 2019-11-25T15:33:40.424 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:40.419+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:40.503 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:40.501+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:40.718 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:40.714+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:40.865 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:40.861+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:40.891 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:40.885+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:41.169 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:41.164+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:41.211 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:41.206+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:41.468 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:41.463+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:41.540 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:41.536+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:41.691 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:41.688+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:41.870 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:41.867+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:41.936 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:41.931+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:42.138 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:42.130+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:42.235 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:42.229+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:42.481 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:42.477+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:42.581 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:42.567+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:42.702 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:42.698+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:42.865 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:42.856+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:42.895 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:42.890+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:43.156 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:43.149+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:43.226 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:43.220+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:43.483 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:43.478+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:43.616 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:43.611+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:43.669 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:43.666+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:43.831 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:43.827+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:43.934 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:43.928+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:44.109 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:44.104+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:44.274 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:44.268+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:44.491 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:44.485+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:44.600 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:44.596+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:33:44.672 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:33:44.668+0000 16949700 -1 osd.1 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:25.953974+0000 front 2019-11-25T15:32:25.922794+0000 (oldest deadline 2019-11-25T15:33:11.165717+0000) 2019-11-25T15:33:44.795 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:33:44.791+0000 16949700 -1 osd.2 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:28.259777+0000 front 2019-11-25T15:32:28.265536+0000 (oldest deadline 2019-11-25T15:33:11.151597+0000) 2019-11-25T15:33:44.985 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:33:44.980+0000 16949700 -1 osd.5 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.803455+0000 front 2019-11-25T15:32:29.123637+0000 (oldest deadline 2019-11-25T15:33:13.788238+0000) 2019-11-25T15:33:45.086 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:33:45.081+0000 16949700 -1 osd.7 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:30.096309+0000 front 2019-11-25T15:32:25.195157+0000 (oldest deadline 2019-11-25T15:33:09.859975+0000) 2019-11-25T15:33:45.235 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:33:45.229+0000 16949700 -1 osd.4 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:29.937992+0000 front 2019-11-25T15:32:29.922165+0000 (oldest deadline 2019-11-25T15:33:10.930745+0000) 2019-11-25T15:33:45.422 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:33:45.416+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 44sec e183 v183) 2019-11-25T15:33:45.453 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:33:45.448+0000 16949700 -1 osd.6 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.771927+0000 front 2019-11-25T15:32:26.720918+0000 (oldest deadline 2019-11-25T15:33:11.306159+0000) 2019-11-25T15:33:45.625 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:33:45.617+0000 16949700 -1 osd.0 183 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:32:26.245815+0000 front 2019-11-25T15:32:26.194944+0000 (oldest deadline 2019-11-25T15:33:10.871544+0000) 2019-11-25T15:34:19.537 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:19.530+0000 16949700 -1 osd.5 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:20.569 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:20.565+0000 16949700 -1 osd.5 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:21.536 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:21.532+0000 16949700 -1 osd.5 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:22.515 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:22.511+0000 16949700 -1 osd.5 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:23.524 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:23.520+0000 16949700 -1 osd.5 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:24.540 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:24.536+0000 16949700 -1 osd.5 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:25.524 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:25.520+0000 16949700 -1 osd.5 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:26.517 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:26.514+0000 16949700 -1 osd.5 187 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:26.685 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:26.658+0000 38a77700 -1 osd.2 188 build_incremental_map_msg missing incremental map 188 2019-11-25T15:34:26.792 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:26.751+0000 36272700 -1 osd.2 188 build_incremental_map_msg missing incremental map 188 2019-11-25T15:34:26.801 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:26.787+0000 124aa700 -1 osd.5 188 build_incremental_map_msg missing incremental map 188 2019-11-25T15:34:27.006 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:26.875+0000 36a73700 -1 osd.2 188 build_incremental_map_msg missing incremental map 188 2019-11-25T15:34:27.158 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:27.081+0000 36a73700 -1 osd.5 188 build_incremental_map_msg missing incremental map 188 2019-11-25T15:34:27.600 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:27.590+0000 16949700 -1 osd.5 189 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:28.587 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:28.583+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:29.578 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:29.574+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:30.585 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:30.581+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:31.569 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:31.565+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:32.589 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:32.580+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:33.571 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:33.567+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:34.563 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:34.558+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:35.619 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:35.615+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:36.595 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:36.592+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:37.600 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:37.597+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:38.640 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:38.637+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:39.616 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:39.614+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:40.598 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:40.595+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:41.598 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:41.595+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:42.005 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:19:46.588 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.025 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:19:46.632 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.028 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:19:46.636 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.031 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:19:46.639 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.034 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:19:46.642 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.037 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:19:46.621 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.183 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:19:46.790 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.187 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:19:46.795 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.254 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:19:46.838 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:34:42.568 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:42.565+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:43.396 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T15:34:43.351+0000 20e21700 -1 RGWWatcher::handle_error cookie 363659392 err (107) Transport endpoint is not connected 2019-11-25T15:34:43.528 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:43.525+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:44.183 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:44.180+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:44.536 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:44.532+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:44.538 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:44.535+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:44.829 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:44.825+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:44.938 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:44.934+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:45.173 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:45.170+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:45.232 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:45.229+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:45.534 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:45.529+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:45.534 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:45.531+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:45.878 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:45.873+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:45.880 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:45.876+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:45.976 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:45.972+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:46.201 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:46.199+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:46.215 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:46.212+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:46.579 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:46.575+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:46.581 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:46.578+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:46.897 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:46.894+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:46.902 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:46.899+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:46.955 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:46.950+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:47.175 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:47.172+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:47.188 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:47.185+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:47.610 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:47.605+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:47.611 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:47.607+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:47.854 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:47.850+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:47.926 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:47.922+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:47.973 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:47.969+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:48.158 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:48.154+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:48.224 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:48.220+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:48.585 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:48.581+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:48.586 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:48.582+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:48.865 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:48.861+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:48.868 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:48.865+0000 16949700 -1 osd.4 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:48.953 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:48.949+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:48.958 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:48.954+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:49.180 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:49.177+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:49.216 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:49.210+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:49.625 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:49.618+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:49.626 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:49.620+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:49.857 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:49.851+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:49.857 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:49.853+0000 16949700 -1 osd.4 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:49.918 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:49.913+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:50.008 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:50.005+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:50.138 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:50.135+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:50.241 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:50.237+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:50.640 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:50.636+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:50.642 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:50.638+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:50.875 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:50.870+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:50.880 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:50.876+0000 16949700 -1 osd.4 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:50.968 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:50.963+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:50.969 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:50.966+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:51.108 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:51.105+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:51.293 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:51.290+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:51.668 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:51.663+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:51.668 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:51.664+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:51.848 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:51.843+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:51.848 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:51.843+0000 16949700 -1 osd.4 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:51.941 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:51.936+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:51.986 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:51.983+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:52.089 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:52.086+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:52.315 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:52.313+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:52.630 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:52.624+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:52.631 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:52.626+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:52.843 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:52.838+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:52.844 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:52.840+0000 16949700 -1 osd.4 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:52.935 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:52.931+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:53.038 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:53.035+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:53.084 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:53.082+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:53.333 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:53.329+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:53.683 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:53.678+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:53.684 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:53.679+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:53.877 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:53.873+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:53.880 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:53.877+0000 16949700 -1 osd.4 190 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:53.971 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:53.967+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:54.057 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:54.054+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:54.075 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:54.072+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:54.387 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:54.382+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:54.659 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:54.654+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:54.663 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:54.659+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:54.927 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:54.922+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:54.929 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:54.925+0000 16949700 -1 osd.4 190 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:54.964 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:54.959+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:55.098 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:55.095+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:55.124 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:55.122+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:55.386 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:55.383+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:55.646 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:55.639+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:55.651 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:55.641+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:55.949 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:55.945+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:55.953 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:55.948+0000 16949700 -1 osd.4 190 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:55.976 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:55.971+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:56.135 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:56.131+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:56.179 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:56.175+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:56.346 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:56.343+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:56.602 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:56.596+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:56.603 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:56.598+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:56.919 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:56.913+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:56.919 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:56.915+0000 16949700 -1 osd.4 190 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:56.991 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:56.987+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:57.149 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:57.146+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:57.151 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:57.148+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:57.306 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:34:57.303+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:34:57.395 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:57.392+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:57.606 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:57.600+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:57.606 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:57.602+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:57.967 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:57.963+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:57.970 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:57.967+0000 16949700 -1 osd.4 190 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:58.016 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:58.011+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:58.162 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:58.155+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:58.199 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:58.196+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:58.322 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:34:58.318+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:34:58.410 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:58.406+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:58.578 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:58.572+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:58.578 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:58.574+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:58.949 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:58.943+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:58.949 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:58.945+0000 16949700 -1 osd.4 190 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:34:58.976 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:58.972+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:34:59.128 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:34:59.123+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:34:59.182 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:34:59.179+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:34:59.360 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:34:59.357+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:34:59.461 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:34:59.457+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:34:59.558 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:59.554+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:34:59.560 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:34:59.557+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:34:59.976 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:59.971+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:34:59.977 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:34:59.972+0000 16949700 -1 osd.4 190 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:00.003 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:34:59.998+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:00.109 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:00.106+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:00.151 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:00.147+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:00.358 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:00.355+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:00.427 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:00.424+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:00.519 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:00.513+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:00.519 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:00.515+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:00.961 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:00.957+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:00.962 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:00.959+0000 16949700 -1 osd.4 190 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:01.005 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:01.002+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:01.075 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:01.073+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:01.129 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:01.126+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:01.349 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:01.345+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:01.457 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:01.453+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:01.518 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:01.513+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:01.519 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:01.515+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:01.932 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:01.925+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:01.934 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:01.929+0000 16949700 -1 osd.4 190 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:02.026 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:02.022+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:02.048 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:02.045+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:02.123 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:02.119+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:02.370 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:02.366+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:02.471 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:02.467+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:02.508 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:02.503+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:02.509 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:02.504+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:02.933 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:02.928+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:02.934 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:02.929+0000 16949700 -1 osd.4 190 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:02.983 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:02.979+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:03.051 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:03.044+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:03.134 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:03.130+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:03.393 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:03.390+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:03.453 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:03.448+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:03.495 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:03.491+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:03.496 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:03.492+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:03.948 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:03.944+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:03.951 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:03.948+0000 16949700 -1 osd.4 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:03.991 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:03.986+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:04.017 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:04.013+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:04.120 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:04.115+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:04.419 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:04.414+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:04.420 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:04.415+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:04.481 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:04.477+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:04.491 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:04.485+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:04.492 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:04.487+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:04.945 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:04.941+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:04.948 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:04.943+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:04.949 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:04.945+0000 16949700 -1 osd.4 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:05.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:05.030+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:05.126 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:05.120+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:05.435 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:05.431+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:05.438 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:05.434+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:05.524 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:05.518+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:05.524 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:05.520+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:05.535 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:05.531+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:05.972 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:05.968+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:05.986 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:05.982+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:05.987 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:05.983+0000 16949700 -1 osd.4 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:06.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:06.028+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:06.082 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:06.078+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:06.419 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:06.414+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:06.420 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:06.415+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:06.497 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:06.492+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:06.497 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:06.493+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:06.566 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:06.560+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:06.936 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:06.930+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:06.991 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:06.987+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:07.029 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:07.024+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:07.032 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:07.028+0000 16949700 -1 osd.4 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:07.047 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:07.043+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:07.390 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:07.386+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:07.393 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:07.390+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:07.488 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:07.483+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:07.491 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:07.487+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:07.583 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:07.579+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:07.927 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:07.922+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:07.961 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:07.958+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:08.027 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:08.023+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:08.086 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:08.081+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:08.087 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:08.083+0000 16949700 -1 osd.4 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:08.440 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:08.431+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:08.442 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:08.433+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:08.534 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:08.528+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:08.534 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:08.530+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:08.609 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:08.606+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:08.890 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:08.885+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:08.953 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:08.950+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:09.074 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:09.071+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:09.097 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:09.093+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:09.101 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:09.097+0000 16949700 -1 osd.4 190 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:09.405 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:09.402+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:09.408 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:09.406+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:09.538 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:09.532+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:09.539 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:09.534+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:09.661 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:09.658+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:09.867 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:09.862+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:09.926 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:09.917+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:10.068 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:10.064+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:10.071 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:10.067+0000 16949700 -1 osd.4 190 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:10.086 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:10.083+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:10.428 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:10.426+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:10.430 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:10.428+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:10.531 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:10.527+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:10.534 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:10.530+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:10.712 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:10.709+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:10.828 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:10.824+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:10.916 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:10.913+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:11.062 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:11.057+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:11.109 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:11.104+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:11.110 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:11.106+0000 16949700 -1 osd.4 190 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:11.429 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:11.426+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:11.432 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:11.429+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:11.565 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:11.560+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:11.567 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:11.564+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:11.697 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:11.694+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:11.792 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:11.788+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:11.921 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:11.918+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:12.037 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:12.033+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:12.118 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:12.112+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:12.119 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:12.114+0000 16949700 -1 osd.4 190 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:12.435 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:12.432+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:12.435 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:12.433+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:12.607 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:12.601+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:12.607 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:12.603+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:12.753 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:12.750+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:12.833 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:12.828+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:12.907 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:12.904+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:13.050 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:13.048+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:13.102 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:13.097+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:13.103 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:13.099+0000 16949700 -1 osd.4 190 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:13.404 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:13.400+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:13.405 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:13.402+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:13.651 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:13.647+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:13.659 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:13.652+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:13.778 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:13.773+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:13.871 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:13.866+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:13.952 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:13.948+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:14.109 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:14.100+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:14.139 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:14.134+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:14.144 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:14.139+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:14.410 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:14.405+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:14.411 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:14.407+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:14.656 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:14.652+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:14.659 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:14.655+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:14.812 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:14.808+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:14.900 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:14.895+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:14.956 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:14.953+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:15.073 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:15.068+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:15.111 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:15.107+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:15.112 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:15.108+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:15.388 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:15.384+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:15.390 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:15.386+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:15.648 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:15.642+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:15.648 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:15.643+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:15.727 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:35:15.715+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e190 v190) 2019-11-25T15:35:15.785 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:15.783+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:15.935 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:15.931+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:15.959 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:15.955+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:16.091 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:16.086+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:16.092 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:16.087+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:16.103 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:16.100+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:16.348 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:16.345+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:16.350 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:16.348+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:16.695 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:16.690+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:16.696 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:16.691+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:16.817 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:16.813+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:16.933 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:16.929+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:16.974 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:16.970+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:17.081 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:17.068+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:17.083 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:17.065+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:17.083 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:17.066+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:17.317 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:17.313+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:17.317 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:17.314+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:17.696 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:17.691+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:17.699 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:17.695+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:17.785 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:17.781+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:17.937 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:17.934+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:17.962 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:17.935+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:18.064 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:18.059+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:18.065 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:18.062+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:18.068 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:18.063+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:18.281 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:18.278+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:18.283 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:18.281+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:18.669 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:18.654+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:18.669 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:18.657+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:18.801 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:18.797+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:18.911 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:18.907+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:18.966 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:18.961+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:19.085 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:19.081+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:19.088 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:19.084+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:19.102 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:19.098+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:19.286 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:19.282+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:19.286 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:19.283+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:19.652 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:19.647+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:19.653 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:19.649+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:19.769 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:19.766+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:19.954 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:19.947+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:19.974 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:19.968+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:20.070 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:20.067+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:20.129 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:20.123+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:20.129 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:20.125+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:20.299 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:20.296+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:20.300 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:20.297+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:20.607 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:20.602+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:20.608 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:20.603+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:20.737 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:35:20.732+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e190 v190) 2019-11-25T15:35:20.800 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:20.797+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:20.907 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:20.904+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:20.975 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:20.970+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:21.026 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:21.023+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:21.180 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:21.174+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:21.180 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:21.176+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:21.274 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:21.270+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:21.276 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:21.272+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:21.621 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:21.616+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:21.623 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:21.619+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:21.830 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:21.826+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:21.953 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:21.950+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:22.005 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:22.001+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:22.073 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:22.070+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:22.209 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:22.203+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:22.210 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:22.205+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:22.291 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:22.286+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:22.292 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:22.288+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:22.643 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:22.638+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:22.644 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:22.639+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:22.837 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:22.833+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:22.970 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:22.967+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:22.989 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:22.984+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:23.085 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:23.082+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:23.175 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:23.170+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:23.179 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:23.174+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:23.322 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:23.319+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:23.323 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:23.320+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:23.656 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:23.651+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:23.656 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:23.652+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:23.827 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:23.824+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:23.953 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:23.949+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:23.980 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:23.976+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:24.069 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:24.066+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:24.141 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:24.135+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:24.142 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:24.138+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:24.346 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:24.343+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:24.349 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:24.346+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:24.707 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:24.703+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:24.710 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:24.707+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:24.880 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:24.876+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:24.971 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:24.967+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:25.010 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:25.005+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:25.091 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:25.086+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:25.105 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:25.099+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:25.105 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:25.101+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:25.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:25.349+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:25.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:25.350+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:25.708 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:25.703+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:25.709 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:25.704+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:25.756 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:35:25.748+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e190 v190) 2019-11-25T15:35:25.923 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:25.920+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:25.930 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:25.924+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:26.052 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:26.047+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:26.105 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:26.096+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:26.117 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:26.112+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:26.118 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:26.114+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:26.369 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:26.366+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:26.373 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:26.370+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:26.757 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:26.754+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:26.760 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:26.757+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:26.925 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:26.921+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:26.930 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:26.927+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:27.017 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:27.013+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:27.061 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:27.053+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:27.149 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:27.146+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:27.150 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:27.147+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:27.333 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:27.328+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:27.333 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:27.330+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:27.798 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:27.794+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:27.799 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:27.796+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:27.958 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:27.954+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:27.963 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:27.959+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:28.004 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:28.000+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:28.103 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:28.100+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:28.180 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:28.176+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:28.181 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:28.178+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:28.310 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:28.307+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:28.315 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:28.311+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:28.838 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:28.834+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:28.838 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:28.835+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:28.932 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:28.929+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:28.995 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:28.992+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:29.042 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:29.039+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:29.100 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:29.098+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:29.165 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:29.162+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:29.167 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:29.164+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:29.329 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:29.324+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:29.329 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:29.327+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:29.818 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:29.814+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:29.819 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:29.816+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:29.953 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:29.950+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:30.002 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:29.999+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:30.055 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:30.051+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:30.121 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:30.117+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:30.213 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:30.202+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:30.213 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:30.203+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:30.359 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:30.354+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:30.360 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:30.356+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:30.773 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:35:30.768+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e190 v190) 2019-11-25T15:35:30.852 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:30.848+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:30.852 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:30.849+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:30.930 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:30.927+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:30.977 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:30.974+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:31.075 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:31.072+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:31.079 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:31.076+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:31.226 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:31.221+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:31.227 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:31.223+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:31.350 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:31.347+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:31.357 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:31.352+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:31.820 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:31.815+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:31.821 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:31.817+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:31.953 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:31.949+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:31.956 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:31.953+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:32.052 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:32.049+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:32.085 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:32.081+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:32.252 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:32.244+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:32.258 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:32.250+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:32.372 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:32.369+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:32.375 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:32.373+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:32.775 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:32.770+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:32.775 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:32.772+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:32.922 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:32.916+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:32.975 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:32.965+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:33.017 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:33.015+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:33.094 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:33.089+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:33.213 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:33.209+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:33.214 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:33.211+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:33.423 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:33.419+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:33.424 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:33.421+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:33.779 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:33.775+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:33.782 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:33.779+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:33.891 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:33.887+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:34.007 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:34.004+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:34.056 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:34.054+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:34.078 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:34.074+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:34.193 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:34.190+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:34.196 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:34.193+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:34.451 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:34.447+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:34.452 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:34.449+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:34.819 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:34.814+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:34.819 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:34.816+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:34.911 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:34.907+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:35.031 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:35.028+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:35.085 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:35.080+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:35.116 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:35:35.113+0000 16949700 -1 osd.7 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:00.542954+0000 front 2019-11-25T15:33:59.503385+0000 (oldest deadline 2019-11-25T15:34:44.672719+0000) 2019-11-25T15:35:35.219 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:35.214+0000 16949700 -1 osd.4 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.494905+0000 front 2019-11-25T15:34:02.941945+0000 (oldest deadline 2019-11-25T15:34:45.692863+0000) 2019-11-25T15:35:35.219 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:35.216+0000 16949700 -1 osd.4 190 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:35.426 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:35.422+0000 16949700 -1 osd.3 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:34:23.529161+0000 (oldest deadline 2019-11-25T15:35:03.529161+0000) 2019-11-25T15:35:35.427 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:35:35.424+0000 16949700 -1 osd.3 190 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e188) 2019-11-25T15:35:35.837 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:35.834+0000 16949700 -1 osd.5 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.872117+0000 front 2019-11-25T15:34:02.891824+0000 (oldest deadline 2019-11-25T15:34:44.358388+0000) 2019-11-25T15:35:35.839 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:35.837+0000 16949700 -1 osd.5 190 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:35.873 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:35:35.864+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e190 v190) 2019-11-25T15:35:35.881 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:35:35.877+0000 16949700 -1 osd.2 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:33:59.202655+0000 front 2019-11-25T15:33:59.825368+0000 (oldest deadline 2019-11-25T15:34:44.140254+0000) 2019-11-25T15:35:36.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:35:36.031+0000 16949700 -1 osd.0 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.988865+0000 front 2019-11-25T15:34:02.990637+0000 (oldest deadline 2019-11-25T15:34:44.353849+0000) 2019-11-25T15:35:36.093 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:35:36.090+0000 16949700 -1 osd.1 190 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:34:02.257227+0000 front 2019-11-25T15:34:02.834833+0000 (oldest deadline 2019-11-25T15:34:44.816143+0000) 2019-11-25T15:35:36.378 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:36.353+0000 36a73700 -1 osd.5 191 build_incremental_map_msg missing incremental map 188 2019-11-25T15:35:36.479 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:36.396+0000 16949700 -1 osd.4 191 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:36.882 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:36.879+0000 16949700 -1 osd.5 191 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:110450 2.1es0 2:7b72abc3:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_25c5c82488504486a3d10bc855275e8e:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~142 in=142b,setxattr user.rgw.manifest (361) in=378b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (55) in=75b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-30dfafcdf788455995979e7b4f4185ad (33) in=85b,setxattr user.rgw.x-amz-meta-3ccb74fea6264b73bdad393bc1122365 (33) in=85b,setxattr user.rgw.x-amz-meta-49a4cb19c4224bc08f04ac456b6517f6 (33) in=85b,setxattr user.rgw.x-amz-meta-53cb25187e4944fe8ea86ade9fbad0ec (33) in=85b,setxattr user.rgw.x-amz-meta-5d3dd1ca8f104f589e47ce5c4c4a30b5 (33) in=85b,setxattr user.rgw.x-amz-meta-b4ed1c51319248a9bc0a39aa218b5d53 (33) in=85b,setxattr user.rgw.x-amz-meta-c53797c3318f4309869d1430ee4e6893 (33) in=85b,setxattr user.rgw.x-amz-meta-cc8fa250740147188b22bda2078ceef5 (33) in=85b,setxattr user.rgw.x-amz-meta-db8eacbfcf2c4f89aada62cefe4a5ff7 (33) in=85b,setxattr user.rgw.x-amz-meta-e81b1eb23e0841d79a17055700221046 (33) in=85b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e186) 2019-11-25T15:35:37.436 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:37.432+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:38.468 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:38.464+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:39.452 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:39.449+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:40.500 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:40.496+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:41.503 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:41.499+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:42.548 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:42.544+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:43.530 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:43.526+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:44.533 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:44.528+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:45.576 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:45.571+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:46.590 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:46.585+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:47.613 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:47.609+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:48.589 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:48.584+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:49.634 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:49.631+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:50.686 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:50.676+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:51.721 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:51.716+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:52.737 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:52.729+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:53.693 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:53.688+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:54.346 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:35:54.342+0000 36a73700 -1 osd.5 192 build_incremental_map_msg missing incremental map 188 2019-11-25T15:35:54.741 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:54.737+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:55.698 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:55.695+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:56.698 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:56.694+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:35:57.720 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:35:57.716+0000 16949700 -1 osd.4 192 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:110499 5.3 5:c609908c:::notify.5:head [watch ping cookie 363659392 gen 1] snapc 0=[] ondisk+write+known_if_redirected e187) 2019-11-25T15:36:25.016 INFO:teuthology.orchestra.run.smithi153.stderr:testMetadataOnPut (test.functional.tests.TestFile) ... ok 2019-11-25T15:37:21.990 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:21.984+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:22.267 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:22.262+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:22.650 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:22.645+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:22.952 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:22.947+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:23.287 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:23.281+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:23.608 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:23.604+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:23.682 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:23.677+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:23.999 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:23.992+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:24.306 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:24.301+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:24.592 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:24.588+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:24.647 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:24.642+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:24.866 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:24.863+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:24.903 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:24.898+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:25.043 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:25.038+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:25.329 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:25.323+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:25.604 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:25.599+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:25.607 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:25.601+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:25.843 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:25.840+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:25.935 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:25.930+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:26.097 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:26.091+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:26.318 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:26.311+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:26.582 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:26.578+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:26.632 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:26.627+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:26.665 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:26.660+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:26.868 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:26.864+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:26.982 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:26.976+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:27.100 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:27.094+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:27.305 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:27.300+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:27.548 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:27.545+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:27.640 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:27.634+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:27.708 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:27.705+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:27.848 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:27.845+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:27.945 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:27.939+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:28.098 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:28.092+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:28.342 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:28.336+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:28.567 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:28.564+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:28.659 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:28.654+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:28.695 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:28.691+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:28.889 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:28.884+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:28.967 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:28.962+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:29.122 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:29.117+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:29.350 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:29.345+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:29.540 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:29.537+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:29.654 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:29.649+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:29.704 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:29.699+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:29.879 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:29.875+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:30.021 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:30.015+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:30.138 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:30.132+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:30.380 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:30.375+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:30.510 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:30.505+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:30.691 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:30.687+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:30.693 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:30.688+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:30.852 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:30.848+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:31.063 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:31.058+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:31.152 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:31.147+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:31.360 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:31.355+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:31.463 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:31.460+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:31.718 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:31.712+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:31.719 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:31.716+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:31.855 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:31.851+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:32.081 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:32.076+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:32.198 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:32.192+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:32.358 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:32.353+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:32.466 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:32.462+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:32.687 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:32.684+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:32.722 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:32.717+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:32.840 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:32.836+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:33.070 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:33.065+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:33.182 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:33.177+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:33.387 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:33.382+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:33.493 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:33.488+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:33.707 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:33.702+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:33.735 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:33.732+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:33.829 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:33.826+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:34.054 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:34.049+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:34.200 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:34.190+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:34.402 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:34.396+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:34.513 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:34.510+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:34.720 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:34.715+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:34.782 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:34.778+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:34.817 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:34.810+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:35.085 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:35.079+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:35.247 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:35.237+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:35.367 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:35.361+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:35.523 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:35.519+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:35.754 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:35.745+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:35.834 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:35.831+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:35.865 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:35.862+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:36.045 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:36.041+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:36.248 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:36.242+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:36.374 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:36.369+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:36.498 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:36.494+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:36.796 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:36.790+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:36.834 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:36.830+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:36.844 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:36.840+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:37.025 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:37.019+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:37.266 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:37.261+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:37.337 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:37.331+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:37.550 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:37.548+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:37.806 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:37.801+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:37.824 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:37.820+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:37.871 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:37.867+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:38.049 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:38.044+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:38.301 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:38.296+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:38.372 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:38.368+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:38.539 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:38.535+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:38.848 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:38.842+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:38.856 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:38.853+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:38.905 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:38.900+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:39.073 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:39.069+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:39.270 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:39.265+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:39.353 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:39.349+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:39.514 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:39.509+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:39.806 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:39.801+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:39.879 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:39.874+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:39.885 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:39.880+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:40.088 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:40.082+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:40.248 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:40.242+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:40.366 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:40.360+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:40.547 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:40.543+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:40.793 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:40.788+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:40.866 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:40.861+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:40.921 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:40.917+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:41.057 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:41.052+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:41.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:41.225+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:41.387 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:41.382+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:41.502 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:41.498+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:41.793 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:41.787+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:41.829 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:41.824+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:41.944 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:41.938+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:42.061 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:42.054+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:42.245 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:42.240+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:42.420 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:42.414+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:42.529 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:42.525+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:42.772 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:42.766+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:42.874 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:42.868+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:42.921 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:42.916+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:43.061 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:43.055+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:43.258 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:43.253+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:43.381 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:43.375+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:43.569 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:43.565+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:43.778 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:43.772+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:43.867 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:43.863+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:43.895 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:43.891+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:44.098 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:44.092+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:44.251 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:44.246+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:44.359 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:44.354+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:44.567 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:44.562+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:44.783 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:44.777+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:44.916 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:44.911+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:44.919 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:44.914+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:45.131 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:45.126+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:45.233 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:45.227+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:45.405 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:45.400+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:45.606 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:45.603+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:45.795 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:45.789+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:45.915 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:45.910+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:45.958 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:45.951+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:46.132 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:46.126+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:46.222 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:46.212+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:46.392 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:46.386+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:46.584 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:46.582+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:46.842 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:46.836+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:46.871 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:46.867+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:46.980 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:46.976+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:47.124 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:47.119+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:47.181 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:47.175+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:47.430 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:47.425+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:47.598 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:47.595+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:47.818 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:47.814+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:47.840 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:47.835+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:47.972 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:47.968+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:48.169 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:48.164+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:48.239 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:48.230+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:48.396 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:48.391+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:48.633 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:48.629+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:48.817 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:48.811+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:48.834 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:48.831+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:49.015 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:49.012+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:49.160 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:49.154+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:49.215 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:49.211+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:49.359 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:49.354+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:49.618 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:49.614+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:49.799 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:49.794+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:49.825 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:49.822+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:49.986 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:49.983+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:50.162 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:50.157+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:50.180 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:50.176+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:50.336 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:50.331+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:50.597 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:50.594+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:50.803 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:50.799+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:50.850 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:50.844+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:50.997 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:50.993+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:51.139 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:51.133+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:51.234 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:51.228+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:51.392 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:51.387+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:51.585 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:51.581+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:51.775 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:51.771+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:51.814 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:51.809+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:52.009 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:52.005+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:52.149 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:52.144+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:52.206 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:52.200+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:52.417 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:52.411+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:52.568 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:52.560+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:52.790 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:52.785+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:52.812 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:52.808+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:53.018 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:53.014+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:53.128 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:53.123+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:53.196 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:53.192+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:53.462 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:53.452+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:53.555 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:53.552+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:53.804 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:53.798+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:53.857 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:53.854+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:54.007 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:54.003+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:54.154 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:54.148+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:54.208 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:54.201+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:54.443 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:54.439+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:54.587 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:54.583+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:54.844 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:54.839+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:54.878 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:54.875+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:54.997 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:54.992+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:55.166 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:55.161+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:55.175 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:55.168+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:55.403 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:55.397+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:55.636 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:55.633+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:37:55.840 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:37:55.835+0000 16949700 -1 osd.7 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.620559+0000 front 2019-11-25T15:36:37.500842+0000 (oldest deadline 2019-11-25T15:37:22.409235+0000) 2019-11-25T15:37:55.883 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:37:55.880+0000 16949700 -1 osd.1 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.044669+0000 front 2019-11-25T15:36:41.015250+0000 (oldest deadline 2019-11-25T15:37:25.760781+0000) 2019-11-25T15:37:55.980 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:37:55.975+0000 16949700 -1 osd.2 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:38.800259+0000 front 2019-11-25T15:36:38.799434+0000 (oldest deadline 2019-11-25T15:37:24.665966+0000) 2019-11-25T15:37:56.169 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:37:56.164+0000 16949700 -1 osd.6 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.102332+0000 front 2019-11-25T15:36:41.114496+0000 (oldest deadline 2019-11-25T15:37:24.536422+0000) 2019-11-25T15:37:56.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:37:56.225+0000 16949700 -1 osd.4 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:41.046249+0000 front 2019-11-25T15:36:41.015990+0000 (oldest deadline 2019-11-25T15:37:21.079788+0000) 2019-11-25T15:37:56.455 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:37:56.450+0000 16949700 -1 osd.5 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.619534+0000 front 2019-11-25T15:36:37.622072+0000 (oldest deadline 2019-11-25T15:37:21.363194+0000) 2019-11-25T15:37:56.508 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:37:56.497+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 42sec e195 v195) 2019-11-25T15:37:56.652 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:37:56.649+0000 16949700 -1 osd.0 195 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:36:37.613932+0000 front 2019-11-25T15:36:37.697867+0000 (oldest deadline 2019-11-25T15:37:23.390438+0000) 2019-11-25T15:38:30.005 INFO:teuthology.orchestra.run.smithi153.stderr:testNameLimit (test.functional.tests.TestFile) ... ok 2019-11-25T15:38:30.038 INFO:teuthology.orchestra.run.smithi153.stderr:testNoContentLengthForPut (test.functional.tests.TestFile) ... ok 2019-11-25T15:38:30.482 INFO:teuthology.orchestra.run.smithi153.stderr:testPostOnFileThatDoesNotExist (test.functional.tests.TestFile) ... ok 2019-11-25T15:38:30.567 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:38:30.559+0000 28256700 -1 reset not still connected to 0x14ae4230 2019-11-25T15:38:31.779 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:38:31.770+0000 3a27a700 -1 osd.5 198 build_incremental_map_msg missing incremental map 188 2019-11-25T15:38:41.887 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:38:41.883+0000 34a6f700 -1 osd.2 201 build_incremental_map_msg missing incremental map 188 2019-11-25T15:38:43.285 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:23:47.869 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:43.295 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:23:47.878 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:43.297 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:23:47.904 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:43.298 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:23:47.905 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:43.299 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:23:47.906 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:43.300 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:23:47.907 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:43.525 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:23:48.132 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:43.545 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:23:48.151 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:43.556 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:23:48.139 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:38:51.698 INFO:teuthology.orchestra.run.smithi153.stderr:testRangedGets (test.functional.tests.TestFile) ... ok 2019-11-25T15:38:52.515 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:38:52.512+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:38:53.494 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:38:53.490+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:38:53.527 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:38:53.522+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:38:54.482 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:38:54.478+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:38:54.494 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:38:54.490+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:38:54.687 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:38:54.684+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:38:55.457 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:38:55.454+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:38:55.481 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:38:55.478+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:38:55.502 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:38:55.498+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:38:55.652 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:38:55.648+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:38:56.469 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:38:56.465+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:38:56.486 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:38:56.482+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:38:56.531 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:38:56.527+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:38:56.644 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:38:56.641+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:38:56.654 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:38:56.651+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:38:57.512 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:38:57.507+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:38:57.521 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:38:57.518+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:38:57.544 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:38:57.540+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:38:57.617 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:38:57.613+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:38:57.678 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:38:57.674+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:38:58.494 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:38:58.490+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:38:58.500 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:38:58.497+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:38:58.566 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:38:58.563+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:38:58.595 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:38:58.592+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:38:58.723 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:38:58.719+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:38:58.746 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:38:58.741+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:38:59.456 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:38:59.453+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:38:59.517 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:38:59.514+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:38:59.554 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:38:59.551+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:38:59.568 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:38:59.565+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:38:59.705 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:38:59.700+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:38:59.763 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:38:59.759+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:00.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:00.471+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:00.536 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:00.532+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:00.544 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:00.540+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:00.553 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:00.549+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:00.664 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:00.660+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:00.752 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:00.748+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:01.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:01.443+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:01.502 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:01.499+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:01.526 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:01.521+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:01.536 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:01.531+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:01.712 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:01.707+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:01.731 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:01.727+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:02.428 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:02.425+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:02.496 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:02.493+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:02.552 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:02.548+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:02.574 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:02.569+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:02.743 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:02.739+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:02.751 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:02.746+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:03.444 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:03.440+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:03.479 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:03.476+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:03.550 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:03.545+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:03.589 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:03.585+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:03.735 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:03.730+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:03.781 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:03.776+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:04.436 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:04.432+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:04.456 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:04.453+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:04.539 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:04.534+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:04.580 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:04.577+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:04.718 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:04.713+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:04.768 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:04.764+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:05.453 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:05.449+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:05.467 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:05.463+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:05.541 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:05.537+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:05.620 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:05.616+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:05.686 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:05.682+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:05.771 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:05.765+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:05.824 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:05.821+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:06.430 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:06.427+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:06.507 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:06.503+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:06.547 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:06.542+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:06.591 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:06.587+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:06.656 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:06.651+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:06.823 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:06.818+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:06.849 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:06.847+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:07.451 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:07.446+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:07.539 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:07.535+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:07.562 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:07.559+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:07.597 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:07.591+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:07.710 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:07.704+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:07.797 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:07.792+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:07.862 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:07.859+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:08.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:08.444+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:08.571 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:08.568+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:08.596 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:08.593+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:08.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:08.599+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:08.736 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:08.730+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:08.829 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:08.825+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:08.912 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:08.909+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:09.401 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:09.398+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:09.545 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:09.541+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:09.596 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:09.593+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:09.607 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:09.603+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:09.763 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:09.758+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:09.884 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:09.880+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:09.907 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:09.905+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:10.451 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:10.448+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:10.554 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:10.550+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:10.607 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:10.603+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:10.610 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:10.605+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:10.794 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:10.788+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:10.928 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:10.922+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:10.955 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:10.952+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:11.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:11.444+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:11.560 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:11.556+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:11.599 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:11.595+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:11.655 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:11.649+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:11.841 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:11.836+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:11.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:11.915+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:11.920 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:11.917+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:11.933 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:11.929+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:12.419 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:12.415+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:12.589 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:12.570+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:12.611 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:12.608+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:12.622 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:12.617+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:12.882 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:12.877+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:12.915 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:12.911+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:12.917 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:12.914+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:12.974 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:12.970+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:13.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:13.442+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:13.579 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:13.575+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:13.604 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:13.601+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:13.609 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T15:39:13.606+0000 20e21700 -1 RGWWatcher::handle_error cookie 1471832640 err (107) Transport endpoint is not connected 2019-11-25T15:39:13.638 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:13.635+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:13.841 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:13.836+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:13.941 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:13.938+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:13.944 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:13.941+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:13.972 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:13.966+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:14.415 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:14.412+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:14.560 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:14.556+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:14.595 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:14.592+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:14.638 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:14.634+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:14.857 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:14.845+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:14.904 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:14.900+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:14.904 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:14.901+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:14.967 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:14.962+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:15.431 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:15.427+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:15.576 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:15.572+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:15.579 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:15.574+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:15.596 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:15.593+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:15.864 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:15.859+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:15.864 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:15.860+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:15.865 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:15.861+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:15.937 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:15.933+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:16.468 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:16.464+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:16.570 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:16.565+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:16.610 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:16.607+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:16.637 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:16.634+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:16.853 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:16.848+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:16.866 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:16.864+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:16.869 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:16.867+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:16.960 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:16.956+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:17.460 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:17.457+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:17.527 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:17.522+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:17.528 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:17.523+0000 16949700 -1 osd.4 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:17.661 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:17.658+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:17.676 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:17.672+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:17.809 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:17.805+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:17.909 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:17.906+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:17.911 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:17.909+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:18.007 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:18.002+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:18.442 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:18.438+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:18.576 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:18.569+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:18.576 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:18.572+0000 16949700 -1 osd.4 201 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:18.671 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:18.667+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:18.714 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:18.710+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:18.775 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:18.771+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:18.923 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:18.919+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:18.926 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:18.923+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:19.055 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:19.051+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:19.444 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:19.440+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:19.623 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:19.617+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:19.624 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:19.619+0000 16949700 -1 osd.4 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:19.700 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:19.695+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:19.722 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:19.718+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:19.830 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:19.826+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:19.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:19.916+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:19.923 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:19.920+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:20.030 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:20.025+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:20.494 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:20.491+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:20.646 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:20.640+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:20.647 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:20.642+0000 16949700 -1 osd.4 201 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:20.688 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:20.684+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:20.696 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:20.693+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:20.862 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:20.858+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:20.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:20.940+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:20.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:20.942+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:21.057 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:21.052+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:21.519 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:21.515+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:21.680 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:21.677+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:21.701 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:21.695+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:21.702 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:21.697+0000 16949700 -1 osd.4 201 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:21.713 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:21.709+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:21.910 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:21.905+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:21.944 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:21.939+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:21.944 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:21.941+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:22.050 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:22.046+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:22.547 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:22.544+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:22.658 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:22.654+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:22.660 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:22.654+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:22.661 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:22.657+0000 16949700 -1 osd.4 201 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:22.731 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:22.728+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:22.897 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:22.892+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:22.968 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:22.965+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:22.969 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:22.966+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:23.091 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:23.086+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:23.557 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:23.552+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:23.643 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:23.639+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:23.649 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:23.643+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:23.649 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:23.645+0000 16949700 -1 osd.4 201 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:23.755 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:23.751+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:23.935 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:23.930+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:23.938 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:23.932+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:23.939 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:23.934+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:24.104 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:24.100+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:24.517 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:24.513+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:24.622 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:24.616+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:24.623 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:24.618+0000 16949700 -1 osd.4 201 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:24.676 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:24.671+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:24.798 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:24.794+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:24.902 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:24.897+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:24.913 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:24.907+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:24.914 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:24.909+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:25.063 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:25.058+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:25.529 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:25.525+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:25.617 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:25.611+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:25.618 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:25.613+0000 16949700 -1 osd.4 201 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:25.656 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:25.652+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:25.820 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:25.816+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:25.897 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:25.893+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:25.898 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:25.895+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:25.955 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:25.950+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:26.062 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:26.058+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:26.578 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:26.572+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:26.615 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:26.610+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:26.652 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:26.647+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:26.653 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:26.648+0000 16949700 -1 osd.4 201 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:26.771 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:39:26.766+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 41sec e201 v201) 2019-11-25T15:39:26.835 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:26.830+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:26.874 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:26.869+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:26.877 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:26.874+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:26.993 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:26.988+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:27.040 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:27.035+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:27.605 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:27.600+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:27.611 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:27.607+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:27.648 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:27.642+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:27.649 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:27.644+0000 16949700 -1 osd.4 201 get_health_metrics reporting 8 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:27.796 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:27.791+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:27.899 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:27.894+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:27.900 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:27.895+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:28.012 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:28.007+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:28.035 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:28.028+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:28.606 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:28.601+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:28.625 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:28.621+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:28.640 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:28.633+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:28.643 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:28.636+0000 16949700 -1 osd.4 201 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:28.777 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:28.772+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:28.868 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:28.864+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:28.871 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:28.868+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:29.051 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:29.046+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:29.064 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:29.060+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:29.606 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:29.603+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:29.641 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:29.637+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:29.683 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:29.678+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:29.687 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:29.683+0000 16949700 -1 osd.4 201 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:29.805 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:29.801+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:29.923 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:29.918+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:29.923 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:29.919+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:30.052 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:30.047+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:30.067 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:30.062+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:30.586 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:30.582+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:30.660 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:30.655+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:30.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:30.660+0000 16949700 -1 osd.4 201 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:30.701 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:30.695+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:30.793 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:30.789+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:30.932 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:30.929+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:30.935 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:30.933+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:31.009 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:31.004+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:31.090 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:31.086+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:31.638 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:31.635+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:31.672 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:31.667+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:31.676 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:31.671+0000 16949700 -1 osd.4 201 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:31.733 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:31.729+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:31.832 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:31.829+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:31.975 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:31.970+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:31.976 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:31.972+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:32.029 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:32.025+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:32.050 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:32.044+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:32.106 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:39:32.103+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 41sec e201 v201) 2019-11-25T15:39:32.213 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:24:36.797 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.229 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:24:36.812 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.309 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:24:36.916 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.312 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:24:36.919 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.314 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:24:36.921 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.333 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:24:36.940 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.346 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:24:36.930 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.452 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:24:37.059 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.457 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:24:37.064 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:39:32.636 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:32.633+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:32.680 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:32.675+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:32.698 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:32.680+0000 16949700 -1 osd.4 201 get_health_metrics reporting 9 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:32.704 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:32.701+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:32.825 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:32.822+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:33.006 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:33.001+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:33.012 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:33.008+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:33.013 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:33.010+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:33.047 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:33.042+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:33.645 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:33.643+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:33.717 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:33.713+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:33.718 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:33.714+0000 16949700 -1 osd.4 201 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:33.759 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:33.751+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:33.759 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:33.755+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:33.792 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:33.788+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:33.997 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:33.991+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:34.055 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:34.051+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:34.056 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:34.053+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:34.092 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:34.087+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:34.691 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:34.687+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:34.699 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:34.693+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:34.700 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:34.695+0000 16949700 -1 osd.4 201 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:34.714 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:34.712+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:34.716 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:34.714+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:34.752 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:34.749+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:35.055 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:35.048+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:35.071 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:35.068+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:35.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:35.077+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:35.082 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:35.079+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:35.678 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:35.671+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:35.679 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:35.674+0000 16949700 -1 osd.4 201 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:35.689 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:35.685+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:35.690 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:35.687+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:35.744 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:35.740+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:35.792 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:35.789+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:36.018 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:36.013+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:36.096 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:36.092+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:36.123 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:36.119+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:36.123 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:36.120+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:36.655 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:36.651+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:36.658 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:36.654+0000 16949700 -1 osd.4 201 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:36.680 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:36.676+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:36.681 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:36.678+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:36.758 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:36.754+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:36.797 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:36.793+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:37.001 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:36.996+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:37.124 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:37.120+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:37.125 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:37.121+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:37.131 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:37.126+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:37.149 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:39:37.139+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 41sec e201 v201) 2019-11-25T15:39:37.656 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:37.651+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:37.661 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:37.656+0000 16949700 -1 osd.4 201 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:37.710 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:37.706+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:37.710 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:37.707+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:37.727 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:37.724+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:37.843 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:37.840+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:37.973 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:37.968+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:38.122 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:38.118+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:38.124 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:38.121+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:38.125 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:38.122+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:38.635 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:38.630+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:38.638 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:38.632+0000 16949700 -1 osd.4 201 get_health_metrics reporting 10 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:38.694 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:38.690+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:38.695 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:38.691+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:38.758 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:38.755+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:38.891 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:38.887+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:39.021 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:39.018+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:39.168 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:39.165+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:39.179 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:39.174+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:39.179 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:39.176+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:39.598 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:39.592+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:39.600 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:39.595+0000 16949700 -1 osd.4 201 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:39.716 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:39.712+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:39.739 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:39.737+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:39.741 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:39.739+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:39.847 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:39.844+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:39.979 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:39.976+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:40.135 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:40.131+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:40.136 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:40.133+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:40.139 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:40.136+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:40.603 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:40.598+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:40.605 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:40.600+0000 16949700 -1 osd.4 201 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:40.679 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:40.675+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:40.696 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:40.693+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:40.697 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:40.695+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:40.857 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:40.854+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:40.965 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:40.961+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:41.109 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:41.105+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:41.179 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:41.176+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:41.180 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:41.177+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:41.559 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:41.554+0000 16949700 -1 osd.4 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:11.757975+0000 front 2019-11-25T15:38:11.894281+0000 (oldest deadline 2019-11-25T15:38:52.523385+0000) 2019-11-25T15:39:41.560 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:41.556+0000 16949700 -1 osd.4 201 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:41.690 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:39:41.682+0000 16949700 -1 osd.2 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:12.273075+0000 front 2019-11-25T15:38:12.277282+0000 (oldest deadline 2019-11-25T15:38:54.505028+0000) 2019-11-25T15:39:41.723 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:41.720+0000 16949700 -1 osd.1 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.061159+0000 front 2019-11-25T15:38:14.509653+0000 (oldest deadline 2019-11-25T15:38:56.290905+0000) 2019-11-25T15:39:41.724 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:41.722+0000 16949700 -1 osd.1 201 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:41.846 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:39:41.842+0000 16949700 -1 osd.0 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.035316+0000 front 2019-11-25T15:38:14.207479+0000 (oldest deadline 2019-11-25T15:38:52.184891+0000) 2019-11-25T15:39:42.015 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:39:42.011+0000 16949700 -1 osd.5 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.063203+0000 front 2019-11-25T15:38:13.394061+0000 (oldest deadline 2019-11-25T15:38:58.572056+0000) 2019-11-25T15:39:42.095 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:39:42.092+0000 16949700 -1 osd.7 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:38:14.524360+0000 front 2019-11-25T15:38:14.063023+0000 (oldest deadline 2019-11-25T15:38:54.596567+0000) 2019-11-25T15:39:42.202 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:39:42.199+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 41sec e201 v201) 2019-11-25T15:39:42.202 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:42.198+0000 16949700 -1 osd.3 201 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:38:31.766150+0000 (oldest deadline 2019-11-25T15:39:11.766150+0000) 2019-11-25T15:39:42.206 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:39:42.203+0000 16949700 -1 osd.3 201 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e199) 2019-11-25T15:39:42.682 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:42.658+0000 16949700 -1 osd.4 202 get_health_metrics reporting 11 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:42.865 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:42.836+0000 16949700 -1 osd.1 202 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:43.789 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:43.698+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:44.115 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:44.112+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:44.668 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:44.665+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:45.094 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:45.092+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:45.657 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:45.653+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:46.119 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:46.116+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:46.708 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:46.705+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:47.167 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:47.164+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:47.680 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:47.677+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:48.217 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:48.214+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:48.727 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:48.723+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:49.244 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:49.242+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:49.772 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:49.768+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:50.280 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:50.277+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:50.795 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:50.791+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:51.250 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:51.247+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:51.773 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:51.768+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:52.273 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:52.270+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:52.817 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:52.811+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:53.275 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:53.271+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:53.780 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:53.775+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:54.321 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:54.318+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:54.784 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:54.779+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:55.357 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:55.354+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:55.786 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:55.781+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:56.367 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:56.365+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:56.796 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:56.791+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:57.396 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:57.393+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:57.772 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:57.766+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:58.410 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:58.408+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:58.793 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:58.788+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:59.390 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:39:59.387+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:39:59.806 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:39:59.801+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:00.361 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:00.358+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:00.811 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:00.806+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:01.397 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:01.394+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:01.839 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:01.834+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:02.417 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:02.415+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:02.892 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:02.888+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:03.470 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:03.467+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:03.929 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:03.924+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:04.451 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:04.448+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:04.981 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:04.975+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:05.502 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:05.499+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:05.994 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:05.989+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:06.491 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:06.488+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:07.021 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:07.014+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:07.511 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:07.508+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:07.993 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:07.989+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:08.514 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:08.512+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:08.990 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:08.984+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:09.530 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:09.520+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:10.011 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:10.005+0000 16949700 -1 osd.4 203 get_health_metrics reporting 12 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:10.485 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:40:10.482+0000 16949700 -1 osd.1 203 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111102 6.e 6:73d3125c:root::1f3bfae86631470a9cb1682550589307:head [create,call version.set in=58b,writefull 0~210 in=210b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:11.107 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:40:11.093+0000 16949700 -1 osd.4 204 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111056 7.2 7:475be0dc:::2019-11-25-15-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650-37d8a51060774afca5473a4c85e786e7:head [append 0~507 in=507b] snapc 0=[] ondisk+write+known_if_redirected e201) 2019-11-25T15:40:52.610 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:25:57.193 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:40:52.620 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:25:57.204 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:40:52.636 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:25:57.243 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:40:52.670 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:25:57.277 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:40:52.672 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:25:57.279 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:40:52.692 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:25:57.299 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:40:52.766 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:25:57.349 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:40:52.767 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:25:57.374 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:40:52.801 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:25:57.407 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:41:35.295 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:35.291+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:35.808 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:35.799+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:36.335 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:36.331+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:36.781 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:36.774+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:36.996 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:41:36.991+0000 16949700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.651947+0000 front 2019-11-25T15:40:51.706492+0000 (oldest deadline 2019-11-25T15:41:36.750006+0000) 2019-11-25T15:41:37.366 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:37.361+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:37.501 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:37.496+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:37.789 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:37.784+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:38.015 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:41:38.012+0000 16949700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.651947+0000 front 2019-11-25T15:40:51.706492+0000 (oldest deadline 2019-11-25T15:41:36.750006+0000) 2019-11-25T15:41:38.346 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:38.343+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:38.535 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:38.530+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:38.753 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:38.747+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:38.764 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:38.756+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:38.995 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:41:38.992+0000 16949700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.651947+0000 front 2019-11-25T15:40:51.706492+0000 (oldest deadline 2019-11-25T15:41:36.750006+0000) 2019-11-25T15:41:39.351 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:39.345+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:39.399 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:39.394+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:39.570 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:39.564+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:39.783 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:39.777+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:39.789 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:39.784+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:39.931 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:39.926+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:40.009 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:41:40.005+0000 16949700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.651947+0000 front 2019-11-25T15:40:51.706492+0000 (oldest deadline 2019-11-25T15:41:36.750006+0000) 2019-11-25T15:41:40.366 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:40.360+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:40.422 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:40.417+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:40.593 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:40.587+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:40.767 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:40.761+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:40.780 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:40.775+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:40.913 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:40.909+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:40.975 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:41:40.972+0000 16949700 -1 osd.0 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.651947+0000 front 2019-11-25T15:40:51.706492+0000 (oldest deadline 2019-11-25T15:41:36.750006+0000) 2019-11-25T15:41:41.398 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:41.392+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:41.420 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:41.417+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:41.555 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:41.550+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:41.733 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:41.726+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:41.808 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:41.803+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:41.937 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:41.932+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:42.375 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:42.372+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:42.417 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:42.411+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:42.610 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:42.604+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:42.756 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:42.751+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:42.819 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:42.813+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:42.916 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:42.912+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:43.375 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:43.370+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:43.412 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:43.407+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:43.567 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:43.562+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:43.767 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:43.762+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:43.776 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:43.772+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:43.937 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:43.933+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:44.403 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:44.397+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:44.416 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:44.411+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:44.558 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:44.552+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:44.815 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:44.809+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:44.825 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:44.819+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:44.974 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:44.971+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:45.396 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:45.390+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:45.459 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:45.455+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:45.587 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:45.582+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:45.823 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:45.817+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:45.833 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:45.827+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:45.957 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:45.954+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:46.415 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:46.411+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:46.438 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:46.431+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:46.560 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:46.554+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:46.782 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:46.776+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:46.794 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:46.789+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:46.949 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:46.945+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:47.417 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:47.413+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:47.423 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:47.417+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:47.605 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:47.599+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:47.761 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:47.755+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:47.764 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:47.759+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:47.960 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:47.957+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:48.423 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:48.418+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:48.448 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:48.443+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:48.594 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:48.590+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:48.734 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:48.729+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:48.774 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:48.769+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:49.003 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:48.999+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:49.409 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:49.404+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:49.421 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:49.417+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:49.581 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:49.576+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:49.785 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:49.780+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:49.812 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:49.807+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:49.996 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:49.992+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:50.433 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:50.428+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:50.470 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:50.467+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:50.555 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:50.549+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:50.776 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:50.770+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:50.808 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:50.802+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:51.008 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:51.005+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:51.473 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:51.467+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:51.490 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:51.486+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:51.513 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:51.508+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:51.732 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:51.726+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:51.845 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:51.839+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:51.992 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:51.988+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:52.451 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:52.445+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:52.454 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:52.448+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:52.483 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:52.477+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:52.693 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:52.687+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:52.884 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:52.878+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:52.947 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:52.943+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:53.459 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:53.454+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:53.485 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:53.481+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:53.533 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:53.528+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:53.706 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:53.701+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:53.898 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:53.892+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:53.903 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:53.899+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:54.491 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:54.486+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:54.492 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:54.486+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:54.531 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:54.525+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:54.667 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:54.661+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:54.922 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:54.916+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:54.933 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:54.930+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:55.486 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:55.480+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:55.522 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:55.518+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:55.524 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:55.519+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:55.698 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:55.693+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:55.896 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:55.891+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:55.898 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:55.893+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:56.483 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:56.477+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:56.485 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:56.481+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:56.499 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:56.494+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:56.703 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:56.698+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:56.880 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:56.875+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:56.916 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:56.910+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:57.458 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:57.454+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:57.505 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:57.500+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:57.533 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:57.528+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:57.661 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:57.655+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:57.835 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:57.831+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:57.891 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:57.886+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:58.460 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:58.455+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:58.505 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:58.500+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:58.509 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:58.504+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:58.626 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:58.621+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:58.825 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:58.822+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:58.940 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:58.934+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:41:59.495 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:41:59.489+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:41:59.501 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:41:59.498+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:41:59.514 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:41:59.509+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:41:59.612 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:41:59.607+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:41:59.826 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:41:59.823+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:41:59.974 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:41:59.968+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:00.468 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:00.463+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:00.494 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:00.489+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:00.511 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:00.507+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:00.592 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:00.587+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:00.787 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:00.784+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:00.951 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:00.945+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:01.477 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:01.472+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:01.522 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:01.517+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:01.542 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:01.537+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:01.565 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:01.561+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:01.780 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:01.775+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:01.950 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:01.945+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:02.489 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:02.483+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:02.525 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:02.519+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:02.537 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:02.531+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:02.596 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:02.591+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:02.813 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:02.808+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:02.940 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:02.934+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:03.526 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:03.521+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:03.529 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:03.524+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:03.543 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:03.537+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:03.611 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:03.606+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:03.844 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:03.838+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:03.915 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:03.909+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:04.487 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:04.482+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:04.492 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:04.487+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:04.524 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:04.519+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:04.603 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:04.597+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:04.870 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:04.865+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:04.934 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:04.928+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:05.454 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:05.449+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:05.476 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:05.471+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:05.546 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:05.541+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:05.638 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:05.630+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:05.901 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:05.896+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:05.953 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:05.948+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:06.472 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:06.467+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:06.505 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:06.500+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:06.525 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:06.521+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:06.633 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:06.627+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:06.854 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:06.849+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:06.939 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:06.934+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:07.479 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:07.473+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:07.507 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:07.502+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:07.544 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:07.539+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:07.672 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:07.667+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:07.778 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:42:07.772+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 42sec e207 v207) 2019-11-25T15:42:07.884 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:07.879+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:07.980 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:07.974+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:08.456 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:08.450+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:08.563 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:08.559+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:08.597 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:08.591+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:08.667 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:08.662+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:08.876 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:08.871+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:08.995 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:08.990+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:09.459 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:09.453+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:09.610 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:09.606+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:09.633 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:09.628+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:09.671 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:09.666+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:09.833 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:09.829+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:09.964 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:09.960+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:10.483 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:10.479+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:10.609 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:10.604+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:10.614 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:10.611+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:10.660 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:10.656+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:10.837 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:10.832+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:11.000 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:10.995+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:11.517 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:11.513+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:11.608 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:11.604+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:11.626 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:11.622+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:11.700 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:11.696+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:11.816 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:11.814+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:12.046 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:42:12.041+0000 16949700 -1 osd.7 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.565982+0000 front 2019-11-25T15:40:53.923520+0000 (oldest deadline 2019-11-25T15:41:38.702023+0000) 2019-11-25T15:42:12.565 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:42:12.561+0000 16949700 -1 osd.5 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.273687+0000 front 2019-11-25T15:40:53.023235+0000 (oldest deadline 2019-11-25T15:41:37.075550+0000) 2019-11-25T15:42:12.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:42:12.600+0000 16949700 -1 osd.4 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.913233+0000 front 2019-11-25T15:40:53.910816+0000 (oldest deadline 2019-11-25T15:41:39.246033+0000) 2019-11-25T15:42:12.626 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:42:12.622+0000 16949700 -1 osd.2 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.556688+0000 front 2019-11-25T15:40:53.597629+0000 (oldest deadline 2019-11-25T15:41:35.135440+0000) 2019-11-25T15:42:12.731 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:42:12.727+0000 16949700 -1 osd.6 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:52.972576+0000 front 2019-11-25T15:40:52.848250+0000 (oldest deadline 2019-11-25T15:41:35.058288+0000) 2019-11-25T15:42:12.870 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:42:12.866+0000 16949700 -1 osd.1 207 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:40:53.920016+0000 front 2019-11-25T15:40:53.915875+0000 (oldest deadline 2019-11-25T15:41:39.813809+0000) 2019-11-25T15:42:12.871 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:42:12.859+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 42sec e207 v207) 2019-11-25T15:42:37.556 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:27:42.139 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:42:37.564 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:27:42.148 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:42:37.580 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:27:42.187 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:42:37.584 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:27:42.191 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:42:37.585 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:27:42.192 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:42:37.592 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:27:42.199 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:42:37.688 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:27:42.271 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:42:37.695 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:27:42.302 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:42:37.751 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:27:42.358 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:43:10.952 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:10.948+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:11.408 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:11.405+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:11.965 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:11.961+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:12.442 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:12.438+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:12.521 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:12.517+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:12.957 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:12.954+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:13.485 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:13.481+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:13.495 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:13.491+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:13.597 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:13.594+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:13.726 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:13.723+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:13.920 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:13.915+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:14.448 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:14.443+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:14.518 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:14.513+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:14.623 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:14.619+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:14.711 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:14.707+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:14.891 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:14.885+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:15.484 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:15.479+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:15.558 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:15.555+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:15.653 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:15.650+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:15.711 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:15.708+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:15.865 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:15.859+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:16.190 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:16.185+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:16.529 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:16.523+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:16.560 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:16.557+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:16.625 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:16.621+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:16.722 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:16.717+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:16.863 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:16.858+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:17.193 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:17.188+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:17.549 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:17.545+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:17.580 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:17.575+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:17.626 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:17.622+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:17.752 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:17.749+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:17.913 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:17.907+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:18.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:18.225+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:18.589 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:18.583+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:18.592 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:18.587+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:18.622 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:18.617+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:18.732 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:18.728+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:18.884 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:18.878+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:19.253 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:19.248+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:19.575 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:19.571+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:19.611 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:19.606+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:19.628 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:19.625+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:19.774 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:19.770+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:19.852 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:19.848+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:20.247 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:20.241+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:20.595 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:20.591+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:20.601 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:20.597+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:20.608 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:20.604+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:20.830 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:20.826+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:20.893 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:20.886+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:21.292 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:21.287+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:21.560 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:21.556+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:21.636 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:21.630+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:21.657 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:21.654+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:21.876 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:21.872+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:21.936 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:21.932+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:22.320 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:22.314+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:22.568 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:22.563+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:22.608 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:22.604+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:22.643 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:22.639+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:22.868 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:22.864+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:22.927 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:22.922+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:23.057 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:23.054+0000 16949700 -1 osd.3 214 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:23.329 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:23.324+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:23.528 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:23.523+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:23.618 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:23.613+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:23.651 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:23.648+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:23.871 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:23.867+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:23.981 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:23.975+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:24.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:24.078+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:24.285 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:24.280+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:24.547 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:24.542+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:24.645 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:24.641+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:24.701 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:24.697+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:24.884 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:24.881+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:24.941 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:24.937+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:25.125 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:25.122+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:25.245 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:25.240+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:25.533 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:25.528+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:25.665 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:25.662+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:25.685 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:25.681+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:25.890 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:25.886+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:25.985 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:25.981+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:26.080 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:26.077+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:26.204 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:26.199+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:26.499 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:26.495+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:26.707 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:26.703+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:26.727 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:26.723+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:26.871 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:26.868+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:27.014 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:27.009+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:27.109 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:27.106+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:27.177 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:27.171+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:27.520 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:27.515+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:27.663 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:27.660+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:27.715 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:27.712+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:27.884 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:27.881+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:27.982 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:27.977+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:28.138 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:28.134+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:28.143 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:28.138+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:28.480 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:28.471+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:28.643 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:28.638+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:28.730 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:28.726+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:28.920 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:28.916+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:28.998 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:28.993+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:29.093 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:29.090+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:29.121 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:29.116+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:29.449 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:29.445+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:29.627 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:29.624+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:29.721 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:29.718+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:29.886 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:29.881+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:29.984 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:29.979+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:30.045 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:30.042+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:30.086 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:30.082+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:30.488 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:30.484+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:30.635 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:30.632+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:30.718 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:30.714+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:30.846 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:30.842+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:31.008 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:31.003+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:31.010 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:31.005+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:31.011 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:31.007+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:31.114 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:31.109+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:31.538 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:31.533+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:31.672 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:31.667+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:31.770 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:31.766+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:31.857 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:31.853+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:32.037 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:32.031+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:32.078 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:32.054+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:32.090 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:32.056+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:32.125 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:32.103+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:32.570 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:32.564+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:32.708 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:32.704+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:32.771 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:32.768+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:32.874 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:32.870+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:33.055 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:33.050+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:33.080 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:33.076+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:33.109 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:33.105+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:33.112 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:33.109+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:33.595 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:33.590+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:33.735 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:33.732+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:33.737 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:33.733+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:33.921 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:33.917+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:34.064 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:34.060+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:34.084 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:34.079+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:34.085 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:34.081+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:34.086 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:34.082+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:34.592 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:34.587+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:34.702 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:34.699+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:34.737 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:34.734+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:34.917 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:34.913+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:35.047 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:35.042+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:35.055 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:35.050+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:35.057 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:35.053+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:35.058 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:35.055+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:35.570 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:35.566+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:35.682 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:35.679+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:35.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:35.761+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:35.926 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:35.923+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:36.042 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:36.038+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:36.053 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:36.048+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:36.105 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:36.102+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:36.110 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:36.107+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:36.593 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:36.589+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:36.713 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:36.709+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:36.796 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:36.792+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:36.961 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:36.957+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:37.055 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:37.050+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:37.073 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:37.069+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:37.145 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:37.141+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:37.146 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:37.143+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:37.563 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:37.559+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:37.749 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:37.745+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:37.823 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:37.818+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:37.994 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:37.991+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:38.068 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:38.063+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:38.084 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:38.079+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:38.128 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:38.123+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:38.129 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:38.125+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:38.584 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:38.580+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:38.747 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:38.744+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:38.850 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:38.845+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:39.027 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:39.022+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:39.031 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:39.027+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:39.061 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:39.057+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:39.107 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:39.103+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:39.108 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:39.104+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:39.583 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:39.578+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:39.713 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:39.709+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:39.861 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:39.857+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:40.029 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:40.024+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:40.034 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:40.030+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:40.075 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:40.070+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:40.083 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:40.080+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:40.084 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:40.081+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:40.637 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:40.632+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:40.738 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:40.734+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:40.918 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:40.911+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:41.023 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:41.019+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:41.030 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:41.025+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:41.121 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:41.117+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:41.121 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:41.118+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:41.124 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:41.119+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:41.663 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:41.657+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:41.719 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:41.715+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:41.919 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:41.915+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:42.058 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:42.055+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:42.059 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:42.051+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:42.124 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:42.119+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:42.171 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:42.168+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:42.172 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:42.169+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:42.648 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:42.643+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:42.749 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:42.745+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:42.898 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:42.894+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:43.055 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:43.052+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:43.062 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:43.057+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:43.130 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:43.125+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:43.218 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:43.215+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:43.221 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:43.218+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:43.609 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:43.605+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:43.786 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:43.782+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:43.859 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:43.856+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:44.037 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:44.034+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:44.089 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:44.083+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:44.134 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:44.129+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:44.186 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:44.180+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:44.188 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:44.182+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:44.636 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:44.631+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:44.829 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:44.826+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:44.915 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:44.910+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:45.003 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:44.999+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:45.069 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:45.063+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:45.104 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:45.099+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:45.196 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:45.193+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:45.199 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:45.196+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:45.655 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:45.651+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:45.870 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:45.865+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:45.959 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:45.955+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:45.986 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:45.982+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:46.083 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:46.078+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:46.117 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:46.112+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:46.238 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:46.235+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:46.240 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:46.237+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:46.654 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:46.649+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:46.844 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:46.839+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:46.999 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:46.995+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:47.038 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:47.034+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:47.043 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:47.038+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:47.087 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:47.081+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:47.246 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:47.241+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:47.247 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:47.243+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:47.614 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:47.609+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:47.881 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:47.876+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:47.981 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:47.976+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:48.051 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:48.046+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:48.070 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:48.065+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:48.075 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:48.070+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:48.285 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:48.280+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:48.285 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:48.281+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:48.522 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:43:48.514+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e214 v214) 2019-11-25T15:43:48.646 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:48.637+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:48.927 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:48.922+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:48.987 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:48.982+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:49.036 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:49.031+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:49.041 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:49.036+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:49.055 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:49.051+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:49.264 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:49.259+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:49.265 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:49.261+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:49.653 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:49.648+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:49.976 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:49.971+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:50.024 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:50.019+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:50.026 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:50.021+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:50.071 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:50.066+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:50.088 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:50.083+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:50.239 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:50.235+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:50.242 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:50.238+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:50.649 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:50.645+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:50.955 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:50.949+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:51.046 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:51.041+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:51.055 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:51.050+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:51.083 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:51.080+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:51.133 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:51.129+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:51.255 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:51.251+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:51.259 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:51.254+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:51.653 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:51.648+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:52.007 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:52.002+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:52.008 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:52.003+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:52.028 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:52.023+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:52.135 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:52.131+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:52.170 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:52.165+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:52.237 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:52.232+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:52.240 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:52.236+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:52.657 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:52.653+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:53.034 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:53.030+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:53.050 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:53.045+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:53.078 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:53.073+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:53.096 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:53.093+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:53.203 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:53.199+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:53.265 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:53.262+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:53.267 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:53.264+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:53.534 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:43:53.527+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e214 v214) 2019-11-25T15:43:53.676 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:53.673+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:54.055 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:54.052+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:54.061 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:54.057+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:54.084 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:54.080+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:54.089 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:54.084+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:54.198 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:54.194+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:54.311 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:54.306+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:54.312 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:54.308+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:54.676 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:54.672+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:55.063 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:55.059+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:55.082 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:55.078+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:55.113 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:55.110+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:55.137 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:55.133+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:55.228 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:55.224+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:55.348 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:55.345+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:55.352 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:55.349+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:55.705 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:55.702+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:56.069 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:56.066+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:56.121 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:56.118+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:56.125 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:56.121+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:56.132 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:56.129+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:56.204 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:56.197+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:56.333 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:56.329+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:56.334 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:56.331+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:56.703 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:56.700+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:57.054 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:57.049+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:57.080 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:57.076+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:57.156 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:57.152+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:57.170 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:57.166+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:57.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:57.227+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:57.357 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:57.352+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:57.358 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:57.354+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:57.759 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:57.755+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:58.018 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:58.015+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:58.060 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:58.056+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:58.145 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:58.140+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:58.147 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:58.144+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:58.251 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:58.246+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:58.316 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:58.313+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:58.319 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:58.316+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:58.544 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:43:58.541+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e214 v214) 2019-11-25T15:43:58.754 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:58.750+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:43:59.039 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:43:59.035+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:43:59.043 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:43:59.039+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:43:59.170 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:43:59.166+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:43:59.183 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:43:59.179+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:43:59.237 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:43:59.233+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:43:59.304 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:59.299+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:43:59.304 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:43:59.301+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:43:59.767 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:43:59.764+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:44:00.070 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:44:00.067+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:44:00.080 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:44:00.077+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:44:00.128 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:44:00.124+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:44:00.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:44:00.193+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:44:00.199 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:44:00.197+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:44:00.270 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:44:00.267+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:44:00.271 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:44:00.268+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:44:00.792 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:44:00.788+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:44:01.107 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:44:01.103+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:44:01.115 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:44:01.112+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:44:01.141 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:44:01.137+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:44:01.175 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:44:01.171+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:44:01.239 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:44:01.235+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:44:01.257 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:44:01.253+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:44:01.257 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:44:01.254+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:44:01.765 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:44:01.761+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:44:02.092 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:44:02.089+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:44:02.117 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:44:02.113+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:44:02.126 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:44:02.123+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:44:02.152 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:44:02.148+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:44:02.227 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:44:02.223+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:44:02.283 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:44:02.278+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:44:02.284 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:44:02.280+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:44:02.743 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:44:02.740+0000 16949700 -1 osd.7 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.821606+0000 front 2019-11-25T15:42:28.384771+0000 (oldest deadline 2019-11-25T15:43:12.373238+0000) 2019-11-25T15:44:03.056 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:44:03.052+0000 16949700 -1 osd.2 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.888963+0000 front 2019-11-25T15:42:29.407434+0000 (oldest deadline 2019-11-25T15:43:12.808525+0000) 2019-11-25T15:44:03.125 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:44:03.121+0000 16949700 -1 osd.4 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:29.457591+0000 front 2019-11-25T15:42:30.925550+0000 (oldest deadline 2019-11-25T15:43:15.334337+0000) 2019-11-25T15:44:03.135 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:44:03.132+0000 16949700 -1 osd.1 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:28.389358+0000 front 2019-11-25T15:42:30.887103+0000 (oldest deadline 2019-11-25T15:43:10.858927+0000) 2019-11-25T15:44:03.163 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:44:03.160+0000 16949700 -1 osd.5 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:27.626979+0000 front 2019-11-25T15:42:33.760957+0000 (oldest deadline 2019-11-25T15:43:10.190343+0000) 2019-11-25T15:44:03.228 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:44:03.225+0000 16949700 -1 osd.0 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:42:30.436698+0000 front 2019-11-25T15:42:29.064294+0000 (oldest deadline 2019-11-25T15:43:13.399149+0000) 2019-11-25T15:44:03.265 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:44:03.261+0000 16949700 -1 osd.3 214 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:42:50.861960+0000 (oldest deadline 2019-11-25T15:43:30.861960+0000) 2019-11-25T15:44:03.265 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:44:03.262+0000 16949700 -1 osd.3 214 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e212) 2019-11-25T15:44:03.602 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:44:03.598+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e214 v214) 2019-11-25T15:45:37.655 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:37.651+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:37.868 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:37.863+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:37.898 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:37.894+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:38.637 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:38.633+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:38.849 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:38.845+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:38.863 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:38.858+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:39.669 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:39.664+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:39.810 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:39.805+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:39.872 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:39.866+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:40.528 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:40.523+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:40.726 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:40.721+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:40.789 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:40.786+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:40.830 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:40.825+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:40.992 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:40.987+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:41.546 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:41.542+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:41.563 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:41.559+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:41.629 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:41.624+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:41.736 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:41.732+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:41.780 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:41.775+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:41.874 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:41.869+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:41.992 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:41.989+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:42.534 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:42.530+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:42.586 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:42.581+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:42.587 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:42.583+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:42.734 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:42.729+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:42.771 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:42.766+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:42.845 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:42.840+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:43.007 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:43.002+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:43.536 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:43.533+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:43.575 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:43.569+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:43.576 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:43.572+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:43.712 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:43.708+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:43.760 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:43.756+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:43.898 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:43.894+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:44.004 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:44.000+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:44.509 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:44.505+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:44.562 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:44.556+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:44.623 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:44.617+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:44.728 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:44.724+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:44.765 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:44.760+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:44.872 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:44.867+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:45.026 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:45.022+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:45.483 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:45.479+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:45.519 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:45.514+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:45.595 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:45.590+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:45.746 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:45.741+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:45.810 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:45.805+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:45.847 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:45.842+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:46.008 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:46.004+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:46.502 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:46.495+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:46.503 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:46.498+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:46.572 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:46.567+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:46.771 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:46.767+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:46.829 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:46.823+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:46.900 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:46.895+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:46.982 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:46.977+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:47.509 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:47.505+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:47.539 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:47.533+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:47.586 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:47.581+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:47.777 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:47.773+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:47.850 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:47.844+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:47.952 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:47.947+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:48.009 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:48.005+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:48.530 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:48.524+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:48.542 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:48.539+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:48.614 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:48.609+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:48.814 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:48.810+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:48.841 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:48.837+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:48.977 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:48.971+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:49.026 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:49.023+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:49.501 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:49.495+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:49.506 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:49.501+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:49.642 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:49.637+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:49.801 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:49.796+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:49.840 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:49.835+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:49.941 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:49.935+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:50.043 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:50.040+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:50.477 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:50.471+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:50.510 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:50.505+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:50.667 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:50.660+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:50.803 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:50.798+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:50.870 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:50.865+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:50.949 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:50.943+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:51.087 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:51.083+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:51.526 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:51.520+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:51.538 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:51.534+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:51.707 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:51.701+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:51.765 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:51.760+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:51.855 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:51.849+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:51.915 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:51.908+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:52.086 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:52.080+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:52.526 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:52.521+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:52.549 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:52.545+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:52.679 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:52.674+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:52.808 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:52.804+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:52.826 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:52.821+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:52.870 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:52.865+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:53.051 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:53.048+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:53.490 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:53.485+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:53.599 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:53.594+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:53.662 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:53.655+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:53.794 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:53.788+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:53.821 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:53.817+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:53.867 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:53.859+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:54.094 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:54.090+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:54.464 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:54.458+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:54.612 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:54.608+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:54.700 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:54.694+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:54.788 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:54.783+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:54.867 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:54.863+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:54.894 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:54.887+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:55.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:55.099+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:55.432 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:55.427+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:55.662 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:55.658+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:55.732 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:55.726+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:55.768 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:55.763+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:55.862 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:55.859+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:55.914 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:55.908+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:56.149 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:56.144+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:56.411 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:56.406+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:56.622 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:56.618+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:56.745 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:56.740+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:56.795 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:56.790+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:56.885 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:56.879+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:56.900 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:56.896+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:57.132 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:57.128+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:57.420 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:57.415+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:57.658 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:57.651+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:57.762 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:57.744+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:57.762 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:57.748+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:57.894 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:57.887+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:57.900 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:57.895+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:58.178 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:58.173+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:58.414 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:58.408+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:58.629 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:58.625+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:58.719 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:58.713+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:58.743 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:58.739+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:58.884 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:58.879+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:45:58.938 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:58.932+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:59.138 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:45:59.134+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:45:59.429 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:45:59.424+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:45:59.604 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:45:59.600+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:45:59.722 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:45:59.716+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:45:59.789 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:45:59.783+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:45:59.913 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:45:59.908+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:45:59.925 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:45:59.921+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:00.124 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:00.119+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:00.428 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:00.423+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:00.656 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:00.653+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:00.741 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:00.735+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:00.838 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:00.832+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:00.910 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:00.904+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:00.949 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:00.945+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:01.087 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:01.083+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:01.486 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:01.478+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:01.695 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:01.691+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:01.753 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:01.747+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:01.813 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:01.808+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:01.929 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:01.923+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:01.964 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:01.960+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:02.133 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:02.128+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:02.497 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:02.492+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:02.665 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:02.660+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:02.731 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:02.726+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:02.867 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:02.862+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:02.919 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:02.915+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:02.949 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:02.944+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:03.104 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:03.101+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:03.513 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:03.507+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:03.632 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:03.629+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:03.732 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:03.726+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:03.860 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:03.854+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:03.938 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:03.934+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:03.945 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:03.939+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:04.111 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:04.108+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:04.539 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:04.533+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:04.684 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:04.680+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:04.757 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:04.752+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:04.885 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:04.879+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:04.981 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:04.976+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:04.985 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:04.980+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:05.070 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:05.065+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:05.576 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:05.570+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:05.654 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:05.650+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:05.787 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:05.780+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:05.846 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:05.841+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:05.985 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:05.981+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:06.016 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:06.010+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:06.060 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:06.057+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:06.595 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:06.589+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:06.632 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:06.625+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:06.799 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:06.793+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:06.804 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:06.798+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:06.999 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:06.995+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:07.047 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:07.042+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:07.075 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:07.071+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:07.592 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:07.586+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:07.594 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:07.591+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:07.786 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:07.781+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:07.826 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:07.820+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:08.046 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:08.043+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:08.049 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:08.045+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:08.066 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:08.061+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:08.595 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:08.590+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:08.610 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:08.600+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:08.745 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:08.740+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:08.866 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:08.858+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:09.068 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:09.063+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:09.075 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:09.070+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:09.099 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:09.096+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:09.630 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:09.624+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:09.632 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:09.629+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:09.798 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:09.792+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:09.863 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:09.857+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:10.057 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:10.053+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:10.118 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:10.113+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:10.119 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:10.113+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:10.652 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:10.648+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:10.671 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:10.665+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:10.752 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:10.747+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:10.843 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:10.838+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:11.072 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:11.067+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:11.121 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:11.116+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:11.165 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:11.160+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:11.613 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:11.609+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:11.713 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:11.708+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:11.730 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:11.724+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:11.845 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:11.840+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:12.121 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:12.116+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:12.140 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:12.137+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:12.146 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:12.142+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:12.667 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:12.663+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:12.692 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:12.687+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:12.695 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:12.689+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:12.865 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:12.860+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:13.150 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:13.145+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:13.161 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:13.157+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:13.199 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:46:13.194+0000 16949700 -1 osd.7 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.494150+0000 front 2019-11-25T15:44:54.386214+0000 (oldest deadline 2019-11-25T15:45:37.385712+0000) 2019-11-25T15:46:13.686 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:46:13.679+0000 16949700 -1 osd.6 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.209625+0000 front 2019-11-25T15:44:55.159893+0000 (oldest deadline 2019-11-25T15:45:41.033798+0000) 2019-11-25T15:46:13.707 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:46:13.703+0000 16949700 -1 osd.1 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.513144+0000 front 2019-11-25T15:44:56.681062+0000 (oldest deadline 2019-11-25T15:45:40.145142+0000) 2019-11-25T15:46:13.717 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:46:13.712+0000 16949700 -1 osd.4 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:57.519358+0000 front 2019-11-25T15:44:58.744722+0000 (oldest deadline 2019-11-25T15:45:41.009518+0000) 2019-11-25T15:46:13.843 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:46:13.837+0000 16949700 -1 osd.5 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.613765+0000 front 2019-11-25T15:44:57.490219+0000 (oldest deadline 2019-11-25T15:45:37.373687+0000) 2019-11-25T15:46:14.090 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:46:14.078+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 46sec e220 v220) 2019-11-25T15:46:14.163 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:46:14.157+0000 16949700 -1 osd.0 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:55.592853+0000 front 2019-11-25T15:44:55.348111+0000 (oldest deadline 2019-11-25T15:45:40.033771+0000) 2019-11-25T15:46:14.187 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:46:14.183+0000 16949700 -1 osd.2 220 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:44:53.832427+0000 front 2019-11-25T15:44:53.820477+0000 (oldest deadline 2019-11-25T15:45:37.630069+0000) 2019-11-25T15:46:30.325 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:31:34.908 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:46:30.336 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:31:34.919 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:46:30.347 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:31:34.954 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:46:30.352 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:31:34.959 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:46:30.353 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:31:34.960 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:46:30.360 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:31:34.967 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:46:30.590 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:31:35.197 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:46:30.657 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:31:35.240 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:46:30.671 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:31:35.278 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:47:13.198 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:13.194+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:13.633 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:13.629+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:13.723 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:13.717+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:14.170 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:14.166+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:14.632 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:14.628+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:14.734 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:14.729+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:15.191 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:15.185+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:15.210 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:15.206+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:15.550 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:15.547+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:15.643 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:15.639+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:15.723 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:15.718+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:16.173 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:16.170+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:16.189 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:16.186+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:16.592 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:16.588+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:16.654 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:16.646+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:16.747 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:16.744+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:17.139 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:17.135+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:17.171 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:17.166+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:17.371 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:17.365+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:17.615 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:17.611+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:17.625 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:17.621+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:17.757 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:17.752+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:18.162 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:18.158+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:18.174 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:18.171+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:18.390 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:18.385+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:18.592 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:18.587+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:18.614 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:18.611+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:18.784 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:18.780+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:19.158 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:19.154+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:19.172 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:19.168+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:19.400 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:19.396+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:19.581 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:19.577+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:19.661 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:19.656+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:19.795 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:19.790+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:20.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:20.193+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:20.199 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:20.195+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:20.429 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:20.423+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:20.539 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:20.534+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:20.625 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:20.621+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:20.761 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:20.755+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:21.194 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:21.190+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:21.248 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:21.245+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:21.470 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:21.465+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:21.503 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:21.498+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:21.680 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:21.676+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:21.789 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:21.784+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:22.213 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:22.209+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:22.241 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:22.236+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:22.467 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:22.463+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:22.504 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:22.499+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:22.663 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:22.660+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:22.771 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:22.767+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:23.171 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:23.166+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:23.265 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:23.261+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:23.464 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:23.460+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:23.480 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:23.476+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:23.701 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:23.698+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:23.819 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:23.816+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:24.176 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:24.171+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:24.258 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:24.255+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:24.279 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:24.275+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:24.436 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:24.432+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:24.445 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:24.440+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:24.743 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:24.739+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:24.834 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:24.830+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:25.189 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:25.185+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:25.262 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:25.256+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:25.273 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:25.270+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:25.471 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:25.468+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:25.472 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:25.467+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:25.708 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:25.703+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:25.815 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:25.810+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:26.159 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:26.153+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:26.266 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:26.261+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:26.300 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:26.297+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:26.443 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:26.439+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:26.482 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:26.474+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:26.693 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:26.689+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:26.851 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:26.847+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:27.191 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:27.186+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:27.279 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:27.275+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:27.319 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:27.316+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:27.409 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:27.405+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:27.535 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:27.532+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:27.706 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:27.701+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:27.849 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:27.846+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:28.178 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:28.150+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:28.287 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:28.284+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:28.329 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:28.323+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:28.362 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:28.359+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:28.527 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:28.523+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:28.685 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:28.681+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:28.847 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:28.843+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:29.155 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:29.150+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:29.280 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:29.277+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:29.353 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:29.349+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:29.374 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:29.370+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:29.553 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:29.549+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:29.725 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:29.720+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:29.816 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:29.813+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:30.205 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:30.200+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:30.320 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:30.317+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:30.366 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:30.361+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:30.416 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:30.412+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:30.575 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:30.572+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:30.730 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:30.725+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:30.777 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:30.772+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:31.245 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:31.241+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:31.295 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:31.291+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:31.297 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:31.294+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:31.391 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:31.386+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:31.438 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:31.434+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:31.591 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:31.587+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:31.733 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:31.728+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:31.803 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:31.800+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:32.236 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:32.230+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:32.291 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:32.288+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:32.295 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:32.292+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:32.430 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:32.425+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:32.439 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:32.435+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:32.557 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:32.553+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:32.738 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:32.733+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:32.823 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:32.819+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:33.267 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:33.264+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:33.268 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:33.265+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:33.289 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:33.283+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:33.417 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:33.413+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:33.421 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:33.418+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:33.606 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:33.602+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:33.749 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:33.743+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:33.820 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:33.815+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:34.225 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:34.220+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:34.226 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:34.222+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:34.317 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:34.311+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:34.410 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:34.406+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:34.470 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:34.465+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:34.570 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:34.567+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:34.751 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:34.746+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:34.803 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:34.800+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:35.217 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:35.212+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:35.217 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:35.214+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:35.356 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:35.351+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:35.379 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:35.373+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:35.479 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:35.476+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:35.600 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:35.597+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:35.785 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:35.780+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:35.847 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:35.843+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:36.238 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:36.232+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:36.239 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:36.235+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:36.331 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:36.326+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:36.394 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:36.389+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:36.530 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:36.526+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:36.616 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:36.612+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:36.831 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:36.827+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:36.861 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:36.858+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:37.258 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:37.253+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:37.258 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:37.255+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:37.319 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:37.315+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:37.363 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:37.357+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:37.578 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:37.574+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:37.670 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:37.667+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:37.853 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:37.849+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:37.863 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:37.857+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:38.264 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:38.259+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:38.265 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:38.261+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:38.360 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:38.355+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:38.373 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:38.368+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:38.604 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:38.600+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:38.728 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:38.724+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:38.853 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:38.848+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:38.907 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:38.902+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:39.225 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:39.220+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:39.226 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:39.222+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:39.378 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:39.373+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:39.385 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:39.379+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:39.623 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:39.619+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:39.772 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:39.768+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:39.842 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:39.838+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:39.919 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:39.915+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:40.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:40.197+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:40.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:40.199+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:40.381 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:40.375+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:40.407 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:40.401+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:40.610 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:40.606+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:40.768 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:40.765+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:40.853 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:40.848+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:40.911 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:40.907+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:41.208 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:41.204+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:41.209 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:41.205+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:41.371 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:41.365+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:41.406 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:41.401+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:41.587 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:41.584+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:41.726 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:41.722+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:41.824 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:41.819+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:41.872 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:41.868+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:42.226 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:42.222+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:42.227 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:42.223+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:42.346 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:42.341+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:42.399 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:42.395+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:42.615 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:42.612+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:42.702 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:42.698+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:42.833 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:42.828+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:42.840 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:42.835+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:43.193 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:43.188+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:43.193 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:43.190+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:43.396 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:43.392+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:43.403 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:43.398+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:43.628 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T15:47:43.527+0000 20e21700 -1 RGWWatcher::handle_error cookie 5412299664 err (107) Transport endpoint is not connected 2019-11-25T15:47:43.653 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:43.649+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:43.684 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:43.680+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:43.818 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:43.815+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:43.862 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:43.858+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:44.163 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:44.158+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:44.164 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:44.160+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:44.378 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:44.373+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:44.436 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:44.432+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:44.491 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:47:44.481+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e225 v225) 2019-11-25T15:47:44.681 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:44.677+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:44.683 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:44.680+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:44.837 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:44.833+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:44.914 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:44.909+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:45.144 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:45.140+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:45.147 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:45.144+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:45.412 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:45.408+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:45.452 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:45.448+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:45.652 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:45.649+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:45.721 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:45.711+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:45.851 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:45.847+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:45.929 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:45.923+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:46.151 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:46.147+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:46.152 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:46.148+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:46.390 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:46.385+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:46.467 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:46.462+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:46.616 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:46.611+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:46.668 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:46.664+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:46.848 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:46.845+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:46.947 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:46.942+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:47.128 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:47.124+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:47.129 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:47.125+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:47.352 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:47.348+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:47.432 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:47.427+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:47.436 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:47.431+0000 16949700 -1 osd.4 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:47.657 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:47.654+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:47.719 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:47.716+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:47.859 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:47.856+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:47.912 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:47.908+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:48.099 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:48.094+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:48.100 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:48.096+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:48.384 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:48.379+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:48.475 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:48.469+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:48.475 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:48.470+0000 16949700 -1 osd.4 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:48.643 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:48.639+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:48.768 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:48.764+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:48.891 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:48.888+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:48.922 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:48.917+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:49.102 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:49.097+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:49.103 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:49.099+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:49.405 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:49.401+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:49.444 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:49.440+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:49.447 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:49.443+0000 16949700 -1 osd.4 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:49.508 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:47:49.503+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e225 v225) 2019-11-25T15:47:49.682 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:49.678+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:49.776 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:49.772+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:49.926 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:49.922+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:49.967 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:49.962+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:50.059 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:50.054+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:50.060 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:50.056+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:50.382 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:50.376+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:50.456 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:50.449+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:50.456 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:50.451+0000 16949700 -1 osd.4 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:50.703 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:50.699+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:50.761 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:50.758+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:50.944 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:50.939+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:50.976 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:50.973+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:51.073 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:51.068+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:51.074 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:51.070+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:51.436 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:51.431+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:51.488 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:51.482+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:51.489 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:51.484+0000 16949700 -1 osd.4 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:51.717 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:51.713+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:51.793 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:51.790+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:51.964 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:51.960+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:51.971 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:51.966+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:52.092 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:52.088+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:52.094 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:52.091+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:52.454 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:52.449+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:52.490 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:52.484+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:52.491 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:52.486+0000 16949700 -1 osd.4 225 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:52.730 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:52.726+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:52.830 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:52.826+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:53.003 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:52.999+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:53.010 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:53.006+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:53.122 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:53.116+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:53.122 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:53.117+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:53.410 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:53.405+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:53.457 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:53.452+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:53.458 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:53.454+0000 16949700 -1 osd.4 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:53.708 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:53.704+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:53.836 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:53.832+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:53.993 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:53.987+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:54.011 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:54.006+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:54.090 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:54.086+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:54.093 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:54.090+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:54.431 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:54.425+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:54.456 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:54.450+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:54.456 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:54.452+0000 16949700 -1 osd.4 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:54.535 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:47:54.525+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e225 v225) 2019-11-25T15:47:54.747 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:54.744+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:54.868 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:54.864+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:55.027 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:55.023+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:55.033 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:55.029+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:55.142 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:55.137+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:55.142 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:55.139+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:55.398 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:55.393+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:55.415 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:55.410+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:55.415 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:55.411+0000 16949700 -1 osd.4 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:55.763 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:55.759+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:55.898 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:55.895+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:55.988 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:55.983+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:55.996 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:55.991+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:56.153 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:56.149+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:56.156 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:56.153+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:56.370 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:56.366+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:56.373 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:56.368+0000 16949700 -1 osd.4 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:56.379 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:56.374+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:56.792 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:56.787+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:56.862 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:56.859+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:56.990 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:56.987+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:57.009 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:57.004+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:57.201 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:57.196+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:57.202 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:57.198+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:57.369 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:57.364+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:57.382 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:57.378+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:57.388 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:57.383+0000 16949700 -1 osd.4 225 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:57.842 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:57.839+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:57.873 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:57.869+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:57.971 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:57.965+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:58.049 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:58.045+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:58.166 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:58.161+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:58.176 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:58.170+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:58.363 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:58.358+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:58.366 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:58.362+0000 16949700 -1 osd.4 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:58.414 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:58.409+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:58.837 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:58.833+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:58.912 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:58.909+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:58.997 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:58.993+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:47:59.065 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:47:59.060+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:47:59.181 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:59.177+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:47:59.182 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:47:59.179+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:47:59.388 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:59.384+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:47:59.389 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:47:59.385+0000 16949700 -1 osd.4 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:47:59.456 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:47:59.452+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:47:59.540 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:47:59.536+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e225 v225) 2019-11-25T15:47:59.860 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:47:59.856+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:47:59.910 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:47:59.907+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:47:59.973 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:47:59.968+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:48:00.106 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:48:00.101+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:48:00.223 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:00.220+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:48:00.226 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:00.223+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:48:00.381 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:00.376+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:48:00.382 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:00.377+0000 16949700 -1 osd.4 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:00.429 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:48:00.416+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:48:00.877 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:48:00.872+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:48:00.900 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:48:00.894+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:48:00.962 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:48:00.958+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:48:01.092 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:48:01.086+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:48:01.197 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:01.186+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:48:01.200 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:01.189+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:48:01.345 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:01.336+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:48:01.345 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:01.338+0000 16949700 -1 osd.4 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:01.422 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:48:01.412+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:48:01.839 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:48:01.835+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:48:01.925 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:48:01.921+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:48:01.987 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:48:01.984+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:48:02.080 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:48:02.075+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:48:02.211 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:02.207+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:48:02.213 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:02.210+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:48:02.330 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:02.325+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:48:02.332 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:02.328+0000 16949700 -1 osd.4 225 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:02.401 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:48:02.397+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:48:02.875 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:48:02.870+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:48:02.974 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:48:02.970+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:48:03.015 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:48:03.011+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:48:03.107 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:48:03.103+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:48:03.264 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:03.260+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:48:03.267 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:03.264+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:48:03.292 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:03.287+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:48:03.298 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:03.294+0000 16949700 -1 osd.4 225 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:03.431 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:48:03.427+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:48:03.863 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:48:03.859+0000 16949700 -1 osd.0 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.100356+0000 front 2019-11-25T15:46:30.854434+0000 (oldest deadline 2019-11-25T15:47:13.005008+0000) 2019-11-25T15:48:03.956 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:48:03.952+0000 16949700 -1 osd.2 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.252822+0000 front 2019-11-25T15:46:31.099596+0000 (oldest deadline 2019-11-25T15:47:16.402152+0000) 2019-11-25T15:48:03.984 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:48:03.979+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:48:04.091 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:48:04.087+0000 16949700 -1 osd.7 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.977685+0000 front 2019-11-25T15:46:29.267499+0000 (oldest deadline 2019-11-25T15:47:14.619508+0000) 2019-11-25T15:48:04.298 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:04.293+0000 16949700 -1 osd.3 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:46:50.715799+0000 (oldest deadline 2019-11-25T15:47:30.715799+0000) 2019-11-25T15:48:04.299 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:04.295+0000 16949700 -1 osd.3 225 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:48:04.336 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:04.332+0000 16949700 -1 osd.4 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:28.656290+0000 front 2019-11-25T15:46:28.946446+0000 (oldest deadline 2019-11-25T15:47:12.739800+0000) 2019-11-25T15:48:04.338 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:04.334+0000 16949700 -1 osd.4 225 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:04.407 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:48:04.402+0000 16949700 -1 osd.5 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:29.301004+0000 front 2019-11-25T15:46:29.232163+0000 (oldest deadline 2019-11-25T15:47:14.483526+0000) 2019-11-25T15:48:04.609 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:48:04.603+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e225 v225) 2019-11-25T15:48:04.943 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:48:04.940+0000 16949700 -1 osd.1 225 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:46:31.286670+0000 front 2019-11-25T15:46:31.275802+0000 (oldest deadline 2019-11-25T15:47:13.452602+0000) 2019-11-25T15:48:05.364 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:05.359+0000 16949700 -1 osd.4 226 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:05.649 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:48:05.548+0000 16949700 -1 osd.3 226 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111459 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e224) 2019-11-25T15:48:06.453 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:06.426+0000 16949700 -1 osd.4 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:07.439 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:07.435+0000 16949700 -1 osd.4 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:08.484 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:08.481+0000 16949700 -1 osd.4 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:09.474 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:09.470+0000 16949700 -1 osd.4 227 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:10.468 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:10.464+0000 16949700 -1 osd.4 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:11.516 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:11.512+0000 16949700 -1 osd.4 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:12.531 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:12.528+0000 16949700 -1 osd.4 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:13.509 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:13.505+0000 16949700 -1 osd.4 227 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:14.555 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:14.551+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:15.526 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:15.522+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:16.532 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:16.527+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:17.516 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:17.511+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:18.498 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:18.493+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:19.491 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:19.487+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:20.515 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:20.511+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:21.534 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:21.530+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:22.504 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:22.498+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:23.556 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:23.551+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:24.550 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:24.546+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:25.519 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:25.515+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:26.482 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:26.477+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:48:27.443 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:48:27.439+0000 16949700 -1 osd.4 227 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:111905 5.3 5:c609908c:::notify.5:head [watch ping cookie 5412299664 gen 3] snapc 0=[] ondisk+write+known_if_redirected e225) 2019-11-25T15:49:15.039 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:34:19.623 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:15.099 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:34:19.682 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:15.107 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:34:19.714 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:15.109 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:34:19.716 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:15.112 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:34:19.719 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:15.114 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:34:19.721 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:15.266 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:34:19.873 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:15.270 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:34:19.853 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:15.280 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:34:19.886 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:49:52.252 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:49:52.248+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:49:52.937 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:49:52.931+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:49:53.177 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:49:53.172+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:49:53.217 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:49:53.212+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:49:53.981 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:49:53.977+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:49:54.049 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:49:54.044+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:49:54.140 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:49:54.134+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:49:54.202 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:49:54.199+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:49:54.785 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:49:54.781+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:49:55.023 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:49:55.018+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:49:55.072 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:49:55.066+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:49:55.165 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:49:55.161+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:49:55.198 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:49:55.192+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:49:55.699 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:49:55.694+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:49:55.761 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:49:55.756+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:49:55.890 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:49:55.885+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:49:56.028 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:49:56.024+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:49:56.075 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:49:56.070+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:49:56.142 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:49:56.137+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:49:56.163 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:49:56.158+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:49:56.689 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:49:56.683+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:49:56.736 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:49:56.731+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:49:56.912 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:49:56.906+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:49:57.076 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:49:57.070+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:49:57.119 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:49:57.114+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:49:57.153 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:49:57.148+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:49:57.190 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:49:57.185+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:49:57.695 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:49:57.682+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:49:57.728 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:49:57.724+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:49:57.876 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:49:57.870+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:49:58.058 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:49:58.053+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:49:58.109 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:49:58.105+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:49:58.175 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:49:58.170+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:49:58.201 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:49:58.197+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:49:58.651 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:49:58.646+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:49:58.716 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:49:58.712+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:49:58.888 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:49:58.883+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:49:59.093 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:49:59.087+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:49:59.101 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:49:59.094+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:49:59.170 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:49:59.165+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:49:59.229 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:49:59.224+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:49:59.619 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:49:59.614+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:49:59.772 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:49:59.765+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:49:59.887 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:49:59.882+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:00.109 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:00.103+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:00.149 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:00.144+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:00.155 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:00.150+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:00.253 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:00.247+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:00.643 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:00.637+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:00.822 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:00.816+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:00.935 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:00.931+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:01.119 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:01.112+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:01.123 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:01.118+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:01.161 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:01.157+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:01.277 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:01.272+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:01.686 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:01.681+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:01.837 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:01.831+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:01.904 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:01.900+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:02.105 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:02.099+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:02.161 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:02.155+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:02.210 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:02.205+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:02.246 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:02.241+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:02.700 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:02.694+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:02.807 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:02.801+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:02.921 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:02.914+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:03.131 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:03.125+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:03.147 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:03.142+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:03.258 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:03.254+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:03.288 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:03.283+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:03.751 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:03.745+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:03.854 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:03.849+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:03.951 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:03.947+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:04.113 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:04.108+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:04.126 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:04.122+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:04.248 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:04.244+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:04.340 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:04.337+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:04.743 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:04.737+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:04.855 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:04.849+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:04.981 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:04.975+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:05.124 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:05.118+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:05.175 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:05.170+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:05.289 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:05.284+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:05.304 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:05.300+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:05.721 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:05.697+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:05.902 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:05.896+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:05.995 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:05.991+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:06.109 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:06.104+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:06.171 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:06.165+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:06.291 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:06.286+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:06.306 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:06.302+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:06.707 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:06.702+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:06.933 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:06.928+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:07.019 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:07.011+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:07.149 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:07.144+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:07.205 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:07.196+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:07.319 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:07.316+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:07.334 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:07.330+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:07.759 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:07.753+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:07.907 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:07.902+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:07.978 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:07.974+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:08.139 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:08.134+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:08.170 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:08.166+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:08.295 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:08.291+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:08.377 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:08.372+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:08.758 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:08.752+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:08.923 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:08.918+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:08.969 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:08.965+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:09.115 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:09.105+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:09.153 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:09.147+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:09.310 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:09.305+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:09.341 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:09.338+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:09.811 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:09.805+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:09.968 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:09.963+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:09.998 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:09.994+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:10.070 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:10.063+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:10.155 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:10.149+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:10.317 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:10.314+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:10.362 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:10.357+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:10.823 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:10.817+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:10.959 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:10.954+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:11.031 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:11.027+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:11.117 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:11.110+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:11.121 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:11.116+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:11.294 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:11.290+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:11.323 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:11.319+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:11.851 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:11.846+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:11.918 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:11.912+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:12.036 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:12.032+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:12.115 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:12.109+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:12.155 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:12.149+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:12.316 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:12.308+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:12.317 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:12.314+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:12.814 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:12.808+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:12.902 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:12.897+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:12.997 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:12.993+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:13.102 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:13.097+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:13.186 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:13.181+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:13.286 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:13.282+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:13.305 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:13.301+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:13.838 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:13.833+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:13.930 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:13.925+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:13.974 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:13.970+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:14.112 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:14.106+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:14.181 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:14.176+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:14.242 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:14.238+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:14.301 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:14.298+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:14.895 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:14.890+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:14.897 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:14.892+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:14.982 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:14.978+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:15.117 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:15.112+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:15.204 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:15.200+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:15.218 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:15.212+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:15.338 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:15.334+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:15.888 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:15.882+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:15.937 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:15.931+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:15.955 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:15.951+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:16.157 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:16.151+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:16.225 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:16.220+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:16.227 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:16.224+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:16.291 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:16.288+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:16.931 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:16.926+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:16.935 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:16.929+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:16.942 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:16.937+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:17.119 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:17.113+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:17.265 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:17.260+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:17.277 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:17.271+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:17.278 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:17.273+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:17.889 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:17.885+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:17.925 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:17.919+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:17.927 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:17.920+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:18.101 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:18.096+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:18.238 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:18.235+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:18.263 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:18.259+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:18.271 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:18.265+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:18.887 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:18.882+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:18.920 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:18.914+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:18.974 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:18.968+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:19.089 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:19.083+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:19.207 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:19.203+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:19.237 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:19.232+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:19.261 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:19.255+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:19.885 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:19.880+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:19.906 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:19.903+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:20.010 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:20.001+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:20.097 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:20.091+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:20.218 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:20.215+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:20.254 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:20.250+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:20.299 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:20.294+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:20.916 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:20.911+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:20.940 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:20.936+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:20.971 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:20.966+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:21.102 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:21.096+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:21.220 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:21.216+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:21.277 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:21.272+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:21.281 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:21.275+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:21.938 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:21.932+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:21.947 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:21.943+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:22.026 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:22.020+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:22.155 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:22.150+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:22.247 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:22.243+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:22.288 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:22.284+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:22.301 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:22.296+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:22.915 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:22.909+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:22.959 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:22.954+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:23.072 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:23.066+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:23.208 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:23.203+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:23.253 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:23.249+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:23.262 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:23.259+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:23.265 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:23.261+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:23.872 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:23.867+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:23.919 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:23.915+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:24.121 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:24.115+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:24.201 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:24.196+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:24.260 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:24.256+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:24.264 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:24.259+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:24.268 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:24.264+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:24.885 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:24.880+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:24.893 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:24.888+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:25.083 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:25.077+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:25.166 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:25.160+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:25.246 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:50:25.241+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 43sec e231 v231) 2019-11-25T15:50:25.263 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:25.258+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:25.280 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:25.275+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:25.285 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:25.281+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:25.873 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:25.867+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:25.901 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:25.897+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:26.101 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:26.097+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:26.169 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:26.163+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:26.242 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:26.239+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:26.263 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:26.258+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:26.271 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:26.263+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:26.889 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:26.883+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:26.910 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:26.905+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:27.084 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:27.077+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:27.134 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:27.129+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:27.240 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:27.235+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:27.243 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:27.239+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:27.271 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:27.263+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:27.861 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:27.856+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:27.960 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:27.955+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:28.096 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:28.091+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:28.138 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:28.129+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:28.197 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:28.193+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:28.248 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:28.244+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:28.252 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:28.247+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:28.851 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:28.845+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:28.967 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:28.962+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:29.116 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:29.110+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:29.143 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:29.137+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:29.235 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:29.231+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:29.276 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:29.264+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:29.290 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:29.286+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:29.867 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:50:29.861+0000 16949700 -1 osd.7 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.476658+0000 front 2019-11-25T15:49:10.602010+0000 (oldest deadline 2019-11-25T15:49:54.545532+0000) 2019-11-25T15:50:29.926 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:50:29.921+0000 16949700 -1 osd.1 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.450599+0000 front 2019-11-25T15:49:10.773442+0000 (oldest deadline 2019-11-25T15:49:55.028186+0000) 2019-11-25T15:50:30.085 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:50:30.079+0000 16949700 -1 osd.5 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:10.711812+0000 front 2019-11-25T15:49:10.723751+0000 (oldest deadline 2019-11-25T15:49:52.250454+0000) 2019-11-25T15:50:30.190 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:50:30.184+0000 16949700 -1 osd.4 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.326501+0000 front 2019-11-25T15:49:09.298818+0000 (oldest deadline 2019-11-25T15:49:55.071814+0000) 2019-11-25T15:50:30.239 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:50:30.233+0000 16949700 -1 osd.6 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:08.200810+0000 front 2019-11-25T15:49:08.158069+0000 (oldest deadline 2019-11-25T15:49:53.839313+0000) 2019-11-25T15:50:30.245 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:50:30.241+0000 16949700 -1 osd.2 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:09.299804+0000 front 2019-11-25T15:49:09.301281+0000 (oldest deadline 2019-11-25T15:49:52.558841+0000) 2019-11-25T15:50:30.292 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:50:30.288+0000 16949700 -1 osd.0 231 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:49:11.700439+0000 front 2019-11-25T15:49:09.518470+0000 (oldest deadline 2019-11-25T15:49:51.689507+0000) 2019-11-25T15:50:30.345 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:50:30.327+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 43sec e231 v231) 2019-11-25T15:50:31.676 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:35:36.260 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:50:31.687 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:35:36.270 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:50:31.702 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:35:36.309 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:50:31.704 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:35:36.311 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:50:31.706 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:35:36.311 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:50:31.706 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:35:36.313 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:50:31.793 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:35:36.375 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:50:31.811 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:35:36.418 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:50:31.869 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:35:36.476 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:51:28.625 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:28.621+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:28.858 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:28.853+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:29.669 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:29.666+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:29.889 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:29.885+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:30.698 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:30.694+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:30.754 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:30.750+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:30.871 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:30.865+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:31.747 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:31.742+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:31.792 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:31.788+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:31.802 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:31.798+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:31.867 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:31.863+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:32.591 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:32.587+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:32.749 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:32.745+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:32.794 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:32.790+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:32.799 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:32.793+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:32.914 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:32.909+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:33.578 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:33.574+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:33.733 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:33.728+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:33.772 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:33.769+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:33.790 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:33.786+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:33.895 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:33.891+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:34.561 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:34.557+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:34.735 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:34.731+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:34.785 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:34.779+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:34.786 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:34.783+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:34.891 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:34.886+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:35.569 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:35.564+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:35.733 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:35.729+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:35.754 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:35.750+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:35.804 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:35.801+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:35.891 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:35.888+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:36.618 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:36.614+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:36.754 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:36.750+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:36.773 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:36.769+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:36.805 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:36.801+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:36.850 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:36.847+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:37.622 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:37.617+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:37.735 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:37.731+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:37.775 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:37.770+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:37.794 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:37.790+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:37.859 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:37.855+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:38.322 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:38.318+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:38.637 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:38.632+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:38.751 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:38.746+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:38.795 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:38.789+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:38.818 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:38.814+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:38.878 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:38.874+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:39.303 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:39.300+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:39.622 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:39.617+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:39.768 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:39.762+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:39.782 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:39.778+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:39.783 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:39.780+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:39.845 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:39.841+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:40.321 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:40.312+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:40.651 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:40.646+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:40.754 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:40.749+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:40.820 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:40.816+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:40.827 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:40.822+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:40.876 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:40.870+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:41.346 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:41.341+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:41.699 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:41.694+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:41.731 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:41.726+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:41.797 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:41.792+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:41.815 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:41.811+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:41.886 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:41.881+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:42.365 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:42.361+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:42.691 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:42.686+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:42.723 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:42.720+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:42.811 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:42.806+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:42.817 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:42.813+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:42.903 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:42.898+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:43.412 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:43.408+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:43.668 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:43.663+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:43.755 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:43.751+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:43.775 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:43.769+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:43.791 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:43.786+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:43.894 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:43.889+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:44.374 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:44.369+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:44.654 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:44.648+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:44.734 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:44.729+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:44.741 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:44.736+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:44.801 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:44.796+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:44.893 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:44.888+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:45.399 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:45.393+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:45.400 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:45.395+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:45.688 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:45.684+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:45.701 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:45.697+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:45.784 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:45.779+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:45.844 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:45.840+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:45.937 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:45.933+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:46.356 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:46.351+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:46.357 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:46.353+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:46.693 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:46.688+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:46.736 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:46.733+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:46.809 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:46.804+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:46.857 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:46.854+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:46.954 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:46.950+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:47.360 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:47.357+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:47.364 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:47.361+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:47.662 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:47.656+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:47.734 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:47.730+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:47.783 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:47.777+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:47.839 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:47.836+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:47.972 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:47.969+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:48.375 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:48.371+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:48.378 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:48.375+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:48.672 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:48.666+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:48.755 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:48.751+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:48.824 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:48.819+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:48.841 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:48.838+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:48.988 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:48.983+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:49.347 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:49.342+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:49.348 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:49.345+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:49.690 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:49.686+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:49.710 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:49.707+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:49.780 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:49.776+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:49.886 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:49.881+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:50.039 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:50.035+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:50.312 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:50.307+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:50.313 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:50.309+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:50.701 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:50.696+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:50.707 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:50.704+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:50.767 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:50.762+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:50.908 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:50.905+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:51.034 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:51.030+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:51.364 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:51.360+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:51.366 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:51.363+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:51.679 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:51.673+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:51.692 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:51.688+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:51.808 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:51.802+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:51.865 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:51.861+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:52.008 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:52.005+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:52.326 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:52.322+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:52.329 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:52.326+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:52.667 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:52.662+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:52.736 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:52.732+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:52.808 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:52.804+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:52.920 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:52.915+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:53.024 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:53.021+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:53.293 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:53.288+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:53.293 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:53.289+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:53.662 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:53.656+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:53.750 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:53.745+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:53.812 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:53.806+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:53.878 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:53.875+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:54.004 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:54.001+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:54.278 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:54.273+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:54.278 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:54.275+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:54.684 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:54.678+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:54.772 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:54.766+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:54.813 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:54.807+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:54.922 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:54.919+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:55.027 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:55.022+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:55.242 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:55.239+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:55.245 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:55.242+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:55.730 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:55.726+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:55.775 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:55.772+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:55.806 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:55.800+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:55.905 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:55.902+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:56.064 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:56.061+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:56.246 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:56.242+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:56.247 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:56.244+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:56.703 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:56.698+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:56.822 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:56.819+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:56.854 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:56.849+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:56.925 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:56.921+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:57.048 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:57.044+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:57.240 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:57.235+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:57.241 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:57.237+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:57.751 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:57.745+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:57.833 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:57.828+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:57.860 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:57.854+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:57.928 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:57.924+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:58.091 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:58.087+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:58.259 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:58.255+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:58.261 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:58.258+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:58.757 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:58.752+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:58.855 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:58.850+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:58.898 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:58.894+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:58.909 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:58.905+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:58.909 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:51:58.904+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:51:59.103 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:51:59.098+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:51:59.242 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:59.237+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:51:59.242 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:51:59.238+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:51:59.815 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:51:59.804+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:51:59.888 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:51:59.876+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:51:59.889 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:51:59.878+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:51:59.933 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:51:59.924+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:51:59.954 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:51:59.946+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:00.130 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:00.125+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:00.297 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:00.292+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:00.298 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:00.294+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:00.799 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:52:00.788+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e236 v236) 2019-11-25T15:52:00.859 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:00.855+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:00.896 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:00.891+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:00.901 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:00.896+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:00.957 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:00.952+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:00.972 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:00.968+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:01.099 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:01.095+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:01.275 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:01.271+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:01.279 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:01.275+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:01.818 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:01.812+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:01.918 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:01.914+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:01.918 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:01.914+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:02.000 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:01.989+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:02.013 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:02.010+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:02.094 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:02.090+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:02.313 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:02.308+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:02.314 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:02.310+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:02.792 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:02.786+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:02.906 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:02.900+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:02.918 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:02.913+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:03.002 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:02.997+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:03.064 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:03.060+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:03.109 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:03.105+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:03.345 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:03.340+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:03.346 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:03.342+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:03.755 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:03.750+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:03.886 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:03.879+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:03.928 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:03.925+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:03.983 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:03.978+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:04.072 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:04.068+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:04.160 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:04.157+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:04.203 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T15:52:04.197+0000 20e21700 -1 RGWWatcher::handle_error cookie 1050187904 err (107) Transport endpoint is not connected 2019-11-25T15:52:04.356 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:04.353+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:04.359 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:04.356+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:04.802 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:04.797+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:04.879 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:04.873+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:04.880 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:04.875+0000 16949700 -1 osd.4 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:04.913 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:04.909+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:04.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:04.972+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:05.083 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:05.078+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:05.142 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:05.138+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:05.342 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:05.339+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:05.346 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:05.343+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:05.813 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:52:05.807+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e236 v236) 2019-11-25T15:52:05.840 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:05.836+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:05.888 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:05.883+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:05.889 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:05.884+0000 16949700 -1 osd.4 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:05.944 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:05.931+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:05.949 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:05.944+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:06.077 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:06.073+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:06.134 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:06.131+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:06.386 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:06.381+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:06.387 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:06.383+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:06.826 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:06.821+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:06.882 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:06.876+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:06.883 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:06.878+0000 16949700 -1 osd.4 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:06.995 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:06.991+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:06.996 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:06.991+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:07.120 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:07.115+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:07.181 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:07.174+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:07.435 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:07.430+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:07.436 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:07.432+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:07.863 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:07.858+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:07.916 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:07.911+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:07.917 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:07.912+0000 16949700 -1 osd.4 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:07.959 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:07.953+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:08.014 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:08.010+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:08.145 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:08.141+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:08.224 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:08.220+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:08.454 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:08.449+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:08.455 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:08.451+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:08.874 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:08.868+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:08.874 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:08.870+0000 16949700 -1 osd.4 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:08.895 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:08.889+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:09.014 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:09.007+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:09.062 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:09.059+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:09.155 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:09.151+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:09.225 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:09.221+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:09.439 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:09.435+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:09.440 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:09.436+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:09.887 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:09.883+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:09.902 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:09.897+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:09.905 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:09.901+0000 16949700 -1 osd.4 236 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:10.054 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:10.049+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:10.085 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:10.081+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:10.209 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:10.205+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:10.247 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:10.242+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:10.483 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:10.478+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:10.483 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:10.480+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:10.830 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:52:10.825+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e236 v236) 2019-11-25T15:52:10.852 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:10.848+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:10.871 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:10.867+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:10.873 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:10.870+0000 16949700 -1 osd.4 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:11.070 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:11.066+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:11.102 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:11.097+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:11.210 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:11.207+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:11.297 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:11.293+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:11.485 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:11.480+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:11.486 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:11.482+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:11.871 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:11.865+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:11.871 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:11.866+0000 16949700 -1 osd.4 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:11.879 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:11.874+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:12.101 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:12.097+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:12.114 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:12.109+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:12.222 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:12.218+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:12.307 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:12.304+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:12.483 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:12.479+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:12.485 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:12.482+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:12.837 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:12.831+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:12.839 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:12.835+0000 16949700 -1 osd.4 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:12.931 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:12.926+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:13.147 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:13.142+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:13.154 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:13.150+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:13.234 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:13.231+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:13.343 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:13.339+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:13.459 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:13.454+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:13.459 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:13.456+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:13.888 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:13.883+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:13.891 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:13.887+0000 16949700 -1 osd.4 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:13.923 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:13.918+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:14.190 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:14.184+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:14.207 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:14.201+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:14.288 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:14.285+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:14.383 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:14.379+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:14.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:14.427+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:14.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:14.428+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:14.885 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:14.881+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:14.887 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:14.884+0000 16949700 -1 osd.4 236 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:14.937 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:14.933+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:15.146 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:15.142+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:15.182 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:15.176+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:15.247 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:15.243+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:15.433 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:15.429+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:15.479 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:15.475+0000 16949700 -1 osd.3 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T15:51:05.137457+0000 (oldest deadline 2019-11-25T15:51:45.137457+0000) 2019-11-25T15:52:15.480 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:15.476+0000 16949700 -1 osd.3 236 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:15.903 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:52:15.892+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e236 v236) 2019-11-25T15:52:15.915 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:15.912+0000 16949700 -1 osd.4 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.068881+0000 front 2019-11-25T15:50:47.724794+0000 (oldest deadline 2019-11-25T15:51:30.073436+0000) 2019-11-25T15:52:15.917 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:15.914+0000 16949700 -1 osd.4 236 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:15.979 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:52:15.976+0000 16949700 -1 osd.5 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.713392+0000 front 2019-11-25T15:50:47.726967+0000 (oldest deadline 2019-11-25T15:51:31.668648+0000) 2019-11-25T15:52:16.142 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:52:16.139+0000 16949700 -1 osd.7 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.922314+0000 front 2019-11-25T15:50:47.886272+0000 (oldest deadline 2019-11-25T15:51:58.757128+0000) 2019-11-25T15:52:16.179 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:52:16.176+0000 16949700 -1 osd.1 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:45.655393+0000 front 2019-11-25T15:50:46.189944+0000 (oldest deadline 2019-11-25T15:51:28.067942+0000) 2019-11-25T15:52:16.287 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:52:16.284+0000 16949700 -1 osd.0 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:48.180696+0000 front 2019-11-25T15:50:48.081528+0000 (oldest deadline 2019-11-25T15:51:31.272325+0000) 2019-11-25T15:52:16.400 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:52:16.396+0000 16949700 -1 osd.2 236 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:50:47.698863+0000 front 2019-11-25T15:50:48.340942+0000 (oldest deadline 2019-11-25T15:51:28.563060+0000) 2019-11-25T15:52:16.660 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:52:16.627+0000 16949700 -1 osd.3 237 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e235) 2019-11-25T15:52:16.888 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:16.885+0000 16949700 -1 osd.4 237 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:17.954 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:17.950+0000 16949700 -1 osd.4 238 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:18.922 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:18.919+0000 16949700 -1 osd.4 238 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:19.920 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:19.917+0000 16949700 -1 osd.4 238 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:20.915 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:20.912+0000 16949700 -1 osd.4 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:21.916 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:21.912+0000 16949700 -1 osd.4 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:22.881 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:22.877+0000 16949700 -1 osd.4 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:23.905 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:23.901+0000 16949700 -1 osd.4 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:24.874 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:24.870+0000 16949700 -1 osd.4 238 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:25.836 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:25.832+0000 16949700 -1 osd.4 238 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:26.797 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:26.795+0000 16949700 -1 osd.4 238 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:27.774 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:27.769+0000 16949700 -1 osd.4 238 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:28.782 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:28.777+0000 16949700 -1 osd.4 238 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:29.770 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:29.766+0000 16949700 -1 osd.4 238 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:30.784 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:30.779+0000 16949700 -1 osd.4 238 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:31.813 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:31.807+0000 16949700 -1 osd.4 238 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:32.802 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:32.797+0000 16949700 -1 osd.4 238 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:33.780 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:33.775+0000 16949700 -1 osd.4 238 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:34.782 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:34.778+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:35.837 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:35.831+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:36.860 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:36.856+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:37.857 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:37.852+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:38.850 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:38.845+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:39.857 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:39.853+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:40.863 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:40.858+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:41.352 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:37:45.935 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.403 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:37:45.986 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.462 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:37:46.068 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.487 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:37:46.094 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.491 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:37:46.098 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.496 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:37:46.103 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.633 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:37:46.240 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.655 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:37:46.262 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.712 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:37:46.295 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:52:41.895 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:41.890+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:42.898 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:42.893+0000 16949700 -1 osd.4 238 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:43.914 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:52:43.909+0000 16949700 -1 osd.4 239 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:112302 5.3 5:c609908c:::notify.5:head [watch ping cookie 1050187904 gen 1] snapc 0=[] ondisk+write+known_if_redirected e236) 2019-11-25T15:52:47.948 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:52:47.904+0000 36272700 -1 osd.6 241 build_incremental_map_msg missing incremental map 241 2019-11-25T15:52:48.170 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:52:48.107+0000 38276700 -1 osd.6 241 build_incremental_map_msg missing incremental map 241 2019-11-25T15:52:48.179 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:52:48.121+0000 3a27a700 -1 osd.6 241 build_incremental_map_msg missing incremental map 241 2019-11-25T15:52:48.200 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:52:48.134+0000 3aa7b700 -1 osd.6 241 build_incremental_map_msg missing incremental map 241 2019-11-25T15:52:48.304 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:52:48.155+0000 3aa7b700 -1 osd.6 241 build_incremental_map_msg missing incremental map 241 2019-11-25T15:52:48.313 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:52:48.167+0000 3aa7b700 -1 osd.6 241 build_incremental_map_msg missing incremental map 241 2019-11-25T15:52:48.359 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:52:48.187+0000 3aa7b700 -1 osd.6 241 build_incremental_map_msg missing incremental map 241 2019-11-25T15:52:48.776 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:52:48.641+0000 3b27c700 -1 osd.6 241 build_incremental_map_msg missing incremental map 241 2019-11-25T15:54:08.011 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:08.007+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:08.800 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:08.794+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:09.031 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:09.028+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:09.086 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:09.080+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:09.841 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:09.836+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:09.989 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:09.984+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:10.073 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:10.067+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:10.581 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:10.565+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:10.857 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:10.851+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:11.006 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:11.001+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:11.040 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:11.034+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:11.576 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:11.570+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:11.812 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:11.806+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:11.989 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:11.983+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:12.010 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:12.005+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:12.552 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:12.546+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:12.787 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:12.781+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:12.914 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:12.908+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:13.039 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:13.034+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:13.044 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:13.039+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:13.310 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:13.306+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:13.522 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:13.516+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:13.791 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:13.785+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:13.914 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:13.910+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:14.074 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:14.067+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:14.086 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:14.082+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:14.314 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:14.310+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:14.537 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:14.532+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:14.787 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:14.781+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:14.920 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:14.912+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:15.070 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:15.066+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:15.105 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:15.098+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:15.320 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:15.317+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:15.554 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:15.549+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:15.742 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:15.737+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:15.944 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:15.937+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:16.095 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:16.091+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:16.145 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:16.139+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:16.336 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:16.331+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:16.545 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:16.539+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:16.753 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:16.747+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:16.919 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:16.913+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:17.071 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:17.066+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:17.182 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:17.177+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:17.344 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:17.340+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:17.505 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:17.500+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:17.750 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:17.745+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:17.876 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:17.871+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:18.028 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:18.023+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:18.148 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:18.142+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:18.348 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:18.342+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:18.466 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:18.460+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:18.734 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:18.728+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:18.928 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:18.923+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:19.075 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:19.070+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:19.146 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:19.141+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:19.392 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:19.386+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:19.473 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:19.468+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:19.692 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:19.687+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:19.913 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:19.907+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:20.082 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:20.078+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:20.120 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:20.114+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:20.413 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:20.407+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:20.486 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:20.480+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:20.706 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:20.701+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:20.936 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:20.931+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:21.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:21.061+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:21.080 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:21.075+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:21.397 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:21.392+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:21.514 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:21.509+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:21.744 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:21.739+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:21.962 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:21.956+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:22.039 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:22.034+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:22.070 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:22.064+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:22.419 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:22.413+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:22.535 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:22.529+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:22.737 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:22.731+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:22.956 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:22.950+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:23.030 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:23.026+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:23.052 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:23.046+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:23.462 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:23.456+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:23.573 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:23.566+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:23.689 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:23.685+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:23.957 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:23.952+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:24.009 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:24.005+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:24.031 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:24.026+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:24.498 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:24.495+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:24.587 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:24.582+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:24.727 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:24.722+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:25.002 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:24.998+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:25.003 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:24.998+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:25.054 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:25.048+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:25.543 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:25.539+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:25.572 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:25.566+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:25.707 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:25.700+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:25.975 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:25.971+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:26.004 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:25.998+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:26.019 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:26.013+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:26.527 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:26.523+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:26.546 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:26.540+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:26.700 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:26.695+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:26.948 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:26.945+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:27.030 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:27.023+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:27.048 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:27.042+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:27.501 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:27.497+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:27.594 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:27.588+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:27.661 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:27.655+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:27.954 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:27.949+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:28.003 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:27.998+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:28.030 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:28.025+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:28.525 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:28.521+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:28.558 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:28.552+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:28.705 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:28.700+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:28.961 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:28.955+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:28.985 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:28.981+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:29.056 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:29.050+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:29.547 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:29.542+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:29.589 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:29.584+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:29.690 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:29.684+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:29.926 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:29.921+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:29.973 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:29.969+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:30.019 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:30.014+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:30.506 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:30.502+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:30.618 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:30.612+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:30.672 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:30.667+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:30.888 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:30.882+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:31.010 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:31.005+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:31.028 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:31.023+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:31.461 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:31.457+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:31.636 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:31.631+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:31.640 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:31.635+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:31.913 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:31.907+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:31.997 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:31.991+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:32.035 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:32.030+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:32.428 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:32.424+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:32.623 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:32.618+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:32.634 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:32.629+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:32.880 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:32.875+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:32.964 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:32.958+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:33.064 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:33.059+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:33.446 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:33.442+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:33.597 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:33.592+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:33.607 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:33.603+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:33.932 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:33.926+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:33.977 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:33.970+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:34.089 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:34.086+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:34.470 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:34.465+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:34.601 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:34.595+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:34.605 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:34.600+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:34.966 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:34.961+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:34.989 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:34.984+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:35.052 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:35.048+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:35.474 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:35.471+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:35.636 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:35.630+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:35.638 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:35.633+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:35.986 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:35.980+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:36.006 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:36.000+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:36.069 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:36.065+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:36.454 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:36.445+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:36.665 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:36.659+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:36.669 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:36.664+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:36.841 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:39:41.425 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:36.855 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:39:41.438 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:36.862 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:39:41.469 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:36.869 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:39:41.476 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:36.870 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:39:41.477 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:36.874 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:39:41.481 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:36.983 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:39:41.590 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:36.987 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:36.982+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:37.003 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:39:41.610 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:37.011 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:39:41.594 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:54:37.047 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:37.042+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:37.073 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:37.069+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:37.469 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:37.466+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:37.688 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:37.683+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:37.716 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:37.712+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:37.974 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:37.969+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:38.096 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:38.091+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:38.105 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:38.102+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:38.494 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:38.489+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:38.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:38.659+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:38.690 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:38.684+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:38.936 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:38.930+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:39.065 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:39.060+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:39.066 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:39.061+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:39.515 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:39.511+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:39.652 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:39.648+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:39.694 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:39.690+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:39.961 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:39.956+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:40.046 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:40.040+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:40.107 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:40.103+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:40.529 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:40.525+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:40.646 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:40.640+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:40.669 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:40.664+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:40.770 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:54:40.766+0000 16949700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.711992+0000 front 2019-11-25T15:53:26.496569+0000 (oldest deadline 2019-11-25T15:54:40.688129+0000) 2019-11-25T15:54:40.948 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:40.943+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:41.100 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:41.093+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:41.156 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:41.153+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:41.581 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:41.576+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:41.631 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:41.626+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:41.659 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:41.653+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:41.798 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:54:41.795+0000 16949700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.711992+0000 front 2019-11-25T15:53:26.496569+0000 (oldest deadline 2019-11-25T15:54:40.688129+0000) 2019-11-25T15:54:41.836 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:54:41.832+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 42sec e241 v241) 2019-11-25T15:54:41.990 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:41.984+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:42.076 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:42.070+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:42.152 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:42.147+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:42.600 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:42.591+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:42.636 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:42.631+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:42.692 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:42.687+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:42.776 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:54:42.771+0000 16949700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.711992+0000 front 2019-11-25T15:53:26.496569+0000 (oldest deadline 2019-11-25T15:54:40.688129+0000) 2019-11-25T15:54:43.025 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:43.019+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:43.064 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:43.059+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:43.112 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:43.108+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:43.601 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:43.596+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:43.628 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:43.622+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:43.709 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:43.702+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:43.822 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:54:43.818+0000 16949700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.711992+0000 front 2019-11-25T15:53:26.496569+0000 (oldest deadline 2019-11-25T15:54:40.688129+0000) 2019-11-25T15:54:43.983 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:43.977+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:44.071 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:44.066+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:44.087 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:44.083+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:44.585 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:44.580+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:44.592 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:44.588+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:44.718 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:44.712+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:44.862 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:54:44.857+0000 16949700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.711992+0000 front 2019-11-25T15:53:26.496569+0000 (oldest deadline 2019-11-25T15:54:40.688129+0000) 2019-11-25T15:54:44.965 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:44.959+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:45.073 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:45.067+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:45.078 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:45.074+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:45.585 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:45.581+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:45.623 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:45.618+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:45.739 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:45.734+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:45.913 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:54:45.910+0000 16949700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.711992+0000 front 2019-11-25T15:53:26.496569+0000 (oldest deadline 2019-11-25T15:54:40.688129+0000) 2019-11-25T15:54:45.940 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:45.936+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:46.055 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:54:46.051+0000 16949700 -1 osd.5 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.791233+0000 front 2019-11-25T15:53:26.834385+0000 (oldest deadline 2019-11-25T15:54:08.475558+0000) 2019-11-25T15:54:46.063 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:46.060+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:54:46.574 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:54:46.570+0000 16949700 -1 osd.2 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.406894+0000 front 2019-11-25T15:53:27.472866+0000 (oldest deadline 2019-11-25T15:54:12.617526+0000) 2019-11-25T15:54:46.680 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:54:46.675+0000 16949700 -1 osd.7 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.509283+0000 front 2019-11-25T15:53:27.385620+0000 (oldest deadline 2019-11-25T15:54:08.573194+0000) 2019-11-25T15:54:46.698 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:54:46.694+0000 16949700 -1 osd.4 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:24.456283+0000 front 2019-11-25T15:53:24.920934+0000 (oldest deadline 2019-11-25T15:54:09.705320+0000) 2019-11-25T15:54:46.918 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:54:46.913+0000 16949700 -1 osd.1 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:26.711992+0000 front 2019-11-25T15:53:26.496569+0000 (oldest deadline 2019-11-25T15:54:40.688129+0000) 2019-11-25T15:54:46.930 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:54:46.924+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 42sec e241 v241) 2019-11-25T15:54:46.967 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:54:46.962+0000 16949700 -1 osd.6 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:27.792253+0000 front 2019-11-25T15:53:27.673424+0000 (oldest deadline 2019-11-25T15:54:12.289729+0000) 2019-11-25T15:54:47.076 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:54:47.072+0000 16949700 -1 osd.0 241 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:53:25.608047+0000 front 2019-11-25T15:53:21.377696+0000 (oldest deadline 2019-11-25T15:54:07.813873+0000) 2019-11-25T15:55:23.417 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:55:23.409+0000 11ca9700 -1 osd.1 245 build_incremental_map_msg missing incremental map 245 2019-11-25T15:55:23.668 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:55:23.631+0000 39a79700 -1 osd.1 245 build_incremental_map_msg missing incremental map 245 2019-11-25T15:55:23.829 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:55:23.824+0000 35671700 -1 osd.0 245 build_incremental_map_msg missing incremental map 245 2019-11-25T15:55:23.918 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:55:23.908+0000 37e76700 -1 osd.0 245 build_incremental_map_msg missing incremental map 245 2019-11-25T15:55:24.382 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:55:24.254+0000 3aa7b700 -1 osd.0 245 build_incremental_map_msg missing incremental map 245 2019-11-25T15:55:54.859 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:55:54.855+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:55:55.820 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:55:55.815+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:55:56.792 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:55:56.788+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:55:57.812 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:55:57.809+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:55:58.810 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:55:58.806+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:55:59.847 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:55:59.844+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:00.881 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:00.877+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:01.857 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:01.854+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:02.884 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:02.879+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:03.840 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:03.836+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:04.865 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:04.861+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:05.899 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:05.894+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:06.878 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:06.873+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:07.882 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:07.878+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:08.886 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:08.881+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:09.864 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:09.860+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:10.827 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:10.822+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:11.831 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:11.827+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:12.870 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:12.866+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:13.862 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:13.858+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:14.876 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:14.873+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:15.879 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:15.876+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:16.910 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:16.907+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:17.911 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:17.908+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:18.900 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:18.897+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:19.939 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:19.935+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:20.972 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:20.969+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:21.934 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:21.931+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:22.981 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:22.977+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:23.975 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:23.972+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:24.973 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:24.969+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:25.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:25.962+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:26.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:26.941+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:27.993 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:27.989+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:29.001 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:28.995+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:29.998 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:29.995+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:30.067 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:30.063+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:30.998 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:30.994+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:31.000 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:30.996+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:31.070 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:31.066+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:31.552 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:31.548+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:31.651 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:31.647+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:31.724 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:31.721+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:31.988 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:31.984+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:31.991 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:31.988+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:32.100 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:32.094+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:32.533 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:32.529+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:32.634 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:32.630+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:32.778 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:32.771+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:32.991 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:32.987+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:33.001 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:32.997+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:33.002 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:32.998+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:33.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:33.097+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:33.570 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:33.566+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:33.670 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:33.665+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:33.677 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:33.672+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:33.765 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:33.762+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:33.985 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:33.980+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:34.005 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:34.001+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:34.009 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:34.004+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:34.129 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:34.125+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:34.530 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:34.525+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:34.644 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:34.640+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:34.649 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:34.646+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:34.800 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:34.797+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:35.023 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:35.018+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:35.024 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:35.020+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:35.024 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:35.019+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:35.109 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:35.105+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:35.503 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:35.499+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:35.638 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:35.634+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:35.684 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:35.680+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:35.851 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:35.847+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:35.992 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:35.989+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:36.050 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:36.046+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:36.051 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:36.048+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:36.076 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:36.072+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:36.528 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:36.523+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:36.618 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:36.614+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:36.696 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:36.692+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:36.841 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:36.836+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:36.983 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:36.979+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:37.034 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:37.028+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:37.035 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:37.031+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:37.103 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:37.099+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:37.557 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:37.553+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:37.615 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:37.611+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:37.689 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:37.686+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:37.853 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:37.850+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:37.961 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:37.958+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:38.061 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:38.056+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:38.061 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:38.058+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:38.132 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:38.126+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:38.518 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:38.515+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:38.614 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:38.611+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:38.683 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:38.679+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:38.828 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:38.824+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:38.927 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:38.923+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:39.064 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:39.061+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:39.067 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:39.064+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:39.112 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:39.109+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:39.573 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:39.568+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:39.616 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:39.612+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:39.705 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:39.701+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:39.783 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:39.778+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:39.896 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:39.892+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:40.034 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:40.029+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:40.035 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:40.031+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:40.070 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:40.066+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:40.592 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:40.588+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:40.629 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:40.624+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:40.732 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:40.727+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:40.794 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:40.788+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:40.947 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:40.943+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:41.070 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:41.065+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:41.071 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:41.067+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:41.091 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:41.087+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:41.630 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:41.626+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:41.651 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:41.646+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:41.736 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:41.731+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:41.803 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:41.798+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:41.932 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:41.927+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:42.101 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:42.096+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:42.102 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:42.098+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:42.126 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:42.122+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:42.614 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:42.609+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:42.616 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:42.611+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:42.744 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:42.740+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:42.810 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:42.805+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:42.932 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:42.928+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:43.106 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:43.103+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:43.123 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:43.119+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:43.129 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:43.125+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:43.596 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:43.592+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:43.606 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:43.603+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:43.723 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:43.717+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:43.834 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:43.830+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:43.946 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:43.942+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:44.105 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:44.101+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:44.179 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:44.174+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:44.180 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:44.176+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:44.588 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:44.583+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:44.599 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:44.595+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:44.756 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:44.752+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:44.816 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:44.811+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:44.907 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:44.902+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:45.093 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:45.089+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:45.208 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:45.204+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:45.208 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:45.204+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:45.568 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:45.563+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:45.610 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:45.606+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:45.785 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:45.780+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:45.850 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:45.845+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:45.890 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:45.887+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:46.091 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:46.087+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:46.181 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:46.175+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:46.182 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:46.178+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:46.602 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:46.596+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:46.661 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:46.657+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:46.838 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:46.833+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:46.841 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:46.837+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:46.934 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:46.929+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:47.089 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:47.084+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:47.160 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:47.153+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:47.162 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:47.154+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:47.609 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:47.602+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:47.625 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:47.621+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:47.799 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:47.794+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:47.843 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:47.839+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:47.911 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:47.907+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:48.050 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:48.046+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:48.122 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:48.119+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:48.125 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:48.122+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:48.606 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:48.603+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:48.622 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:48.618+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:48.799 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:48.794+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:48.893 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:48.888+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:48.929 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:48.925+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:49.090 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:49.085+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:49.091 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:49.087+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:49.095 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:49.091+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:49.588 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:49.582+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:49.599 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:49.595+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:49.781 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:49.776+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:49.896 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:49.892+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:49.943 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:49.939+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:50.056 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:50.052+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:50.088 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:50.082+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:50.089 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:50.086+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:50.550 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:50.545+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:50.650 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:50.646+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:50.802 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:50.796+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:50.920 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:50.916+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:50.995 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:50.990+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:51.085 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:51.080+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:51.086 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:51.082+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:51.096 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:51.092+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:51.594 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:51.589+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:51.659 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:51.655+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:51.775 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:51.770+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:51.879 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:51.875+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:52.007 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:52.002+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:52.073 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:52.069+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:52.104 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:52.101+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:52.107 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:52.103+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:52.573 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:52.567+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:52.697 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:52.692+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:52.816 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:52.812+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:52.885 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:52.881+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:52.981 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:52.977+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:53.069 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:53.064+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:53.134 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:53.128+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:53.134 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:53.130+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:53.555 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:53.550+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:53.736 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:53.732+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:53.844 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:53.839+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:53.922 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:53.918+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:53.979 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:53.974+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:54.054 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:54.050+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:54.139 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:54.135+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:54.142 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:54.139+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:54.597 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:54.592+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:54.700 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:54.696+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:54.801 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:54.796+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:54.977 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:54.961+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:54.991 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:54.986+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:55.041 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:55.037+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:55.167 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:55.163+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:55.170 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:55.166+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:55.560 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:55.556+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:55.665 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:55.660+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:55.814 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:55.810+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:55.958 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:55.953+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:55.967 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:55.962+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:56.014 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:56.008+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:56.144 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:56.138+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:56.145 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:56.140+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:56.603 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:56.598+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:56.669 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:56.663+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:56.773 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:56.768+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:56.961 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:56.957+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:56.982 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:56.976+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:57.013 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:57.006+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:57.176 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:57.171+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:57.177 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:57.172+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:57.617 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:57.612+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:57.711 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:57.706+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:57.776 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:57.771+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:57.974 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:57.969+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:57.996 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:57.990+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:58.012 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:58.007+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:58.224 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:58.217+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:58.224 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:58.219+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:58.654 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:58.650+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:58.718 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:58.712+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:58.731 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:58.726+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:56:58.972 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:56:58.967+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:56:59.010 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:59.005+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:56:59.013 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:56:59.008+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:56:59.181 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:59.175+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:56:59.182 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:56:59.177+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:56:59.623 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:56:59.617+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:56:59.694 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:56:59.688+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:56:59.717 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:56:59.712+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:57:00.004 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:56:59.999+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:57:00.015 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:57:00.009+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:57:00.018 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:57:00.013+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:57:00.221 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:57:00.215+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:57:00.222 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:57:00.216+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:57:00.642 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:57:00.637+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:57:00.658 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:57:00.654+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:57:00.702 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:57:00.698+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:57:00.981 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:57:00.976+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:57:01.008 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:57:01.003+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:57:01.057 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:57:01.052+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:57:01.202 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:57:01.197+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:57:01.204 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:57:01.198+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:57:01.666 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:57:01.661+0000 16949700 -1 osd.4 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:45.608598+0000 front 2019-11-25T15:55:45.609734+0000 (oldest deadline 2019-11-25T15:56:30.978231+0000) 2019-11-25T15:57:01.674 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:57:01.670+0000 16949700 -1 osd.5 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.548407+0000 front 2019-11-25T15:55:48.169337+0000 (oldest deadline 2019-11-25T15:56:31.565833+0000) 2019-11-25T15:57:01.684 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:57:01.679+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:57:01.948 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:57:01.942+0000 16949700 -1 osd.0 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:50.825606+0000 front 2019-11-25T15:55:50.823138+0000 (oldest deadline 2019-11-25T15:56:29.506280+0000) 2019-11-25T15:57:01.967 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:57:01.963+0000 16949700 -1 osd.1 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.199571+0000 front 2019-11-25T15:55:48.373235+0000 (oldest deadline 2019-11-25T15:56:30.910556+0000) 2019-11-25T15:57:02.054 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:57:02.049+0000 16949700 -1 osd.7 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:49.920608+0000 front 2019-11-25T15:55:49.919723+0000 (oldest deadline 2019-11-25T15:56:33.372516+0000) 2019-11-25T15:57:02.172 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:57:02.167+0000 16949700 -1 osd.3 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.467355+0000 front 2019-11-25T15:55:47.493116+0000 (oldest deadline 2019-11-25T15:56:32.186429+0000) 2019-11-25T15:57:02.172 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:57:02.168+0000 16949700 -1 osd.3 247 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e245) 2019-11-25T15:57:02.468 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:57:02.462+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 46sec e247 v247) 2019-11-25T15:57:02.727 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:57:02.724+0000 16949700 -1 osd.2 247 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:55:47.681045+0000 front 2019-11-25T15:55:48.002203+0000 (oldest deadline 2019-11-25T15:56:31.102229+0000) 2019-11-25T15:57:11.072 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:57:11.032+0000 34a6f700 -1 osd.6 252 build_incremental_map_msg missing incremental map 252 2019-11-25T15:57:11.196 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:57:11.166+0000 3b27c700 -1 osd.6 252 build_incremental_map_msg missing incremental map 252 2019-11-25T15:57:11.236 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:57:11.185+0000 36272700 -1 osd.6 252 build_incremental_map_msg missing incremental map 252 2019-11-25T15:57:11.236 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:57:11.186+0000 36272700 -1 osd.6 252 build_incremental_map_msg missing incremental map 252 2019-11-25T15:57:11.262 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:57:11.191+0000 39278700 -1 osd.6 252 build_incremental_map_msg missing incremental map 252 2019-11-25T15:57:11.439 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:57:11.259+0000 35a71700 -1 osd.6 252 build_incremental_map_msg missing incremental map 252 2019-11-25T15:57:11.449 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:57:11.301+0000 3b27c700 -1 osd.6 252 build_incremental_map_msg missing incremental map 252 2019-11-25T15:57:37.879 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:42:42.486 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:57:37.900 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:42:42.483 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:57:37.906 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:42:42.489 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:57:37.909 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:42:42.516 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:57:37.943 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:42:42.549 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:57:37.951 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:42:42.557 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:57:38.123 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:42:42.730 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:57:38.145 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:42:42.752 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:57:38.170 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:42:42.753 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:13.279 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:13.273+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:13.372 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:13.367+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:14.096 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:14.092+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:14.238 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:14.234+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:14.348 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:14.345+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:15.145 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:15.140+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:15.195 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:15.192+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:15.383 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:15.378+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:15.513 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:15.506+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:16.137 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:16.133+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:16.157 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:16.151+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:16.197 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:16.192+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:16.406 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:16.401+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:16.512 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:16.507+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:17.132 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:17.127+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:17.188 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:17.183+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:17.199 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:17.196+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:17.426 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:17.423+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:17.455 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:17.450+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:17.525 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:17.521+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:18.129 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:18.123+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:18.231 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:18.225+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:18.252 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:18.248+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:18.337 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:18.332+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:18.417 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:18.412+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:18.476 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:18.470+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:18.536 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:18.531+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:19.166 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:19.162+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:19.228 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:19.223+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:19.296 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:19.291+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:19.303 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:19.298+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:19.434 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:19.429+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:19.451 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:19.447+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:19.529 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:19.524+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:20.180 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:20.176+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:20.255 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:20.251+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:20.270 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:20.265+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:20.328 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:20.323+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:20.395 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:20.390+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:20.446 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:20.442+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:20.573 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:20.569+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:21.140 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:21.134+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:21.286 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:21.281+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:21.288 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:21.283+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:21.339 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:21.335+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:21.446 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:21.441+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:21.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:21.442+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:21.534 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:21.529+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:22.132 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:22.126+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:22.250 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:22.245+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:22.296 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:22.291+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:22.308 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:22.301+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:22.474 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:22.469+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:22.477 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:22.471+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:22.511 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:22.506+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:23.136 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:23.130+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:23.266 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:23.262+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:23.281 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:23.278+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:23.308 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:23.302+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:23.479 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:23.473+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:23.482 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:23.477+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:23.496 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:23.491+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:24.162 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:24.156+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:24.241 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:24.237+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:24.305 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:24.300+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:24.346 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:24.340+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:24.488 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:24.483+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:24.501 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:24.494+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:24.501 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:24.495+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:25.145 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:25.138+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:25.213 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:25.208+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:25.340 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:25.336+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:25.348 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:25.343+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:25.468 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:25.462+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:25.518 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:25.512+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:25.532 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:25.528+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:26.137 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:26.131+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:26.190 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:26.184+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:26.330 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:26.325+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:26.331 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:26.328+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:26.430 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:26.424+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:26.568 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:26.563+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:26.569 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:26.563+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:27.159 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:27.154+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:27.221 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:27.216+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:27.298 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:27.292+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:27.343 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:27.337+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:27.454 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:27.449+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:27.533 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:27.527+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:27.546 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:27.543+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:28.137 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:28.131+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:28.191 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:28.187+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:28.301 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:28.297+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:28.339 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:28.332+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:28.498 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:28.493+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:28.501 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:28.496+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:28.516 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:28.512+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:29.103 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:29.097+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:29.219 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:29.214+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:29.295 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:29.289+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:29.317 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:29.312+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:29.480 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:29.474+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:29.490 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:29.484+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:29.550 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:29.546+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:29.943 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:43:34.526 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:29.952 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:43:34.536 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:29.962 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:43:34.568 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:29.966 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:43:34.573 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:29.968 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:43:34.575 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:29.973 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:43:34.579 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:30.041 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:43:34.648 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:30.046 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:43:34.652 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:30.072 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:43:34.655 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T15:58:30.145 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:30.140+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:30.192 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:30.188+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:30.272 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:30.267+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:30.347 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:30.344+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:30.441 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:30.435+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:30.455 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:30.451+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:30.595 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:30.591+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:31.155 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:31.150+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:31.201 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:31.196+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:31.320 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:31.314+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:31.368 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:31.364+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:31.410 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:31.405+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:31.452 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:31.446+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:31.641 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:31.637+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:32.196 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:32.190+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:32.237 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:32.233+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:32.360 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:32.353+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:32.389 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:32.385+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:32.403 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:32.398+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:32.407 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:32.402+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:32.641 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:32.637+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:33.247 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:33.242+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:33.264 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:33.260+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:33.373 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:33.367+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:33.381 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:33.376+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:33.428 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:33.423+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:33.441 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:33.435+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:33.603 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:33.598+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:34.270 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:34.264+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:34.309 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:34.303+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:34.387 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:34.382+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:34.409 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:34.403+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:34.410 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:34.404+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:34.413 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:34.409+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:34.615 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:34.611+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:35.309 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:35.303+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:35.333 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:35.323+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:35.349 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:35.344+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:35.370 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:35.364+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:35.416 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:35.411+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:35.427 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:35.421+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:35.585 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:35.581+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:36.269 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:36.263+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:36.316 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:36.311+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:36.336 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:36.330+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:36.392 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:36.387+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:36.406 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:36.400+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:36.441 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:36.435+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:36.544 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:36.539+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:37.299 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:37.295+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:37.306 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:37.300+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:37.387 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:37.382+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:37.407 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:37.403+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:37.408 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:37.396+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:37.443 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:37.437+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:37.584 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:37.579+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:38.282 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:38.276+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:38.297 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:38.292+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:38.357 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:38.352+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:38.373 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:38.368+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:38.415 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:38.410+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:38.434 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:38.428+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:38.633 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:38.628+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:39.247 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:39.242+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:39.292 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:39.287+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:39.329 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:39.326+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:39.360 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:39.354+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:39.379 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:39.372+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:39.485 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:39.479+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:39.645 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:39.641+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:40.262 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:40.256+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:40.335 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:40.330+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:40.365 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:40.360+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:40.378 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:40.373+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:40.411 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:40.405+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:40.494 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:40.489+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:40.650 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:40.646+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:41.291 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:41.285+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:41.301 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:41.296+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:41.372 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:41.365+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:41.385 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:41.381+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:41.419 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:41.412+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:41.466 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:41.460+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:41.698 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:41.693+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:42.306 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:42.301+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:42.337 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:42.332+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:42.425 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:42.419+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:42.432 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:42.426+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:42.445 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:42.439+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:42.484 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:42.478+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:42.744 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:42.738+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:43.291 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:43.287+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:43.306 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:43.300+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:43.442 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:43.436+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:43.465 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:43.459+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:43.470 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:43.465+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:43.527 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:43.521+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:43.701 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:43.694+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:44.289 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:44.283+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:44.333 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:44.327+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:44.438 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:44.433+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:44.449 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:44.443+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:44.473 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:44.467+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:44.495 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:44.490+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:44.670 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:44.664+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:45.288 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:45.282+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:45.373 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:45.367+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:45.459 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:45.455+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:45.464 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:45.458+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:45.481 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:45.476+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:45.513 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:45.508+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:45.655 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:45.650+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:46.315 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:46.311+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:46.396 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:46.390+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:46.455 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:46.450+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:46.468 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:46.463+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:46.472 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:46.467+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:46.497 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:46.492+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:46.619 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:46.614+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:47.361 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:47.356+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:47.373 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:47.368+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:47.477 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:47.471+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:47.484 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:47.478+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:47.484 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:47.479+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:47.500 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:47.494+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:47.645 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:47.639+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:47.989 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:58:47.983+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 44sec e252 v252) 2019-11-25T15:58:48.365 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:48.359+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:48.410 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:48.405+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:48.455 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:48.449+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:48.464 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:48.458+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:48.480 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:48.476+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:48.536 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:48.530+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:48.673 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:48.668+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:49.373 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:49.364+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:49.459 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:49.454+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:49.472 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:49.463+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:49.482 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:49.474+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:49.494 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:49.490+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:49.572 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:49.566+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:49.641 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:49.636+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:50.384 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:50.379+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:50.482 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:50.477+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:50.503 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:50.496+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:50.505 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:50.500+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:50.536 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:50.531+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:50.538 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:50.533+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:50.646 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:50.641+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:51.384 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:51.378+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:51.464 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:51.458+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:51.465 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:51.458+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:51.492 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:51.488+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:51.517 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:51.511+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:51.580 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:51.575+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:51.661 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:51.657+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:52.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T15:58:52.379+0000 16949700 -1 osd.4 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.109059+0000 front 2019-11-25T15:57:31.118410+0000 (oldest deadline 2019-11-25T15:58:13.623601+0000) 2019-11-25T15:58:52.429 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:52.423+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:58:52.473 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:58:52.469+0000 16949700 -1 osd.6 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:27.906078+0000 front 2019-11-25T15:57:27.909731+0000 (oldest deadline 2019-11-25T15:58:12.779431+0000) 2019-11-25T15:58:52.477 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:52.472+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:52.489 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T15:58:52.485+0000 16949700 -1 osd.2 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.697220+0000 front 2019-11-25T15:57:31.757099+0000 (oldest deadline 2019-11-25T15:58:17.586912+0000) 2019-11-25T15:58:52.547 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T15:58:52.541+0000 16949700 -1 osd.5 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.059155+0000 front 2019-11-25T15:57:31.026813+0000 (oldest deadline 2019-11-25T15:58:15.831219+0000) 2019-11-25T15:58:52.680 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T15:58:52.676+0000 16949700 -1 osd.0 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.584028+0000 front 2019-11-25T15:57:31.609233+0000 (oldest deadline 2019-11-25T15:58:17.198670+0000) 2019-11-25T15:58:53.070 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T15:58:53.063+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 44sec e252 v252) 2019-11-25T15:58:53.438 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T15:58:53.433+0000 16949700 -1 osd.1 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:29.185158+0000 front 2019-11-25T15:57:29.582808+0000 (oldest deadline 2019-11-25T15:58:12.288358+0000) 2019-11-25T15:58:53.468 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T15:58:53.464+0000 16949700 -1 osd.7 252 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T15:57:31.634298+0000 front 2019-11-25T15:57:31.180570+0000 (oldest deadline 2019-11-25T15:58:14.539204+0000) 2019-11-25T15:59:24.954 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:59:24.572+0000 16949700 -1 osd.6 255 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=32 ondisk+retry+read+known_if_redirected e253) 2019-11-25T15:59:25.626 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:59:25.614+0000 16949700 -1 osd.6 255 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=32 ondisk+retry+read+known_if_redirected e253) 2019-11-25T15:59:26.647 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:59:26.635+0000 16949700 -1 osd.6 256 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=32 ondisk+retry+read+known_if_redirected e253) 2019-11-25T15:59:28.030 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T15:59:27.947+0000 16949700 -1 osd.6 257 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7:e622f76a:gc::gc.20:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=32 ondisk+retry+read+known_if_redirected e253) 2019-11-25T15:59:58.408 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:59:58.404+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T15:59:59.404 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T15:59:59.400+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:00.418 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:00.415+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:01.433 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:01.429+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:02.390 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:02.387+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:03.443 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:03.440+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:04.451 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:04.447+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:05.466 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:05.462+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:06.442 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:06.437+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:07.408 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:07.404+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:08.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:08.425+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:09.433 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:09.429+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:10.390 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:10.387+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:11.441 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:11.437+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:12.465 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:12.461+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:13.502 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:13.498+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:14.510 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:14.506+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:15.556 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:15.551+0000 16949700 -1 osd.3 258 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:16.591 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:16.587+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:17.553 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:17.549+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:18.566 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:18.562+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:19.535 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:19.531+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:20.156 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:20.152+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:20.528 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:20.524+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:20.685 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:20.680+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:20.861 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:20.855+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:21.146 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:21.143+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:21.528 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:21.524+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:21.531 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:21.527+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:21.665 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:21.657+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:21.832 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:21.826+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:22.008 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:22.003+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:22.149 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:22.145+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:22.537 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:22.532+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:22.538 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:22.534+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:22.676 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:22.672+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:22.789 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:22.785+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:23.024 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:23.019+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:23.123 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:23.118+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:23.218 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:23.213+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:23.529 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:23.526+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:23.530 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:23.527+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:23.707 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:23.702+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:23.766 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:23.762+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:23.797 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:23.792+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:24.030 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:24.025+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:24.146 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:24.141+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:24.193 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:24.188+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:24.576 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:24.571+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:24.578 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:24.574+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:24.664 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:24.660+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:24.792 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:24.788+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:24.825 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:24.820+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:25.075 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:25.070+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:25.139 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:25.135+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:25.207 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:25.201+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:25.543 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:25.536+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:25.543 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:25.539+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:25.639 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:25.634+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:25.829 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:25.824+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:25.874 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:25.868+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:26.123 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:26.118+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:26.155 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:26.146+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:26.251 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:26.245+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:26.554 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:26.549+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:26.558 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:26.553+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:26.686 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:26.678+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:26.820 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:26.815+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:26.883 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:26.878+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:27.159 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:27.153+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:27.193 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:27.188+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:27.228 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:27.224+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:27.569 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:27.563+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:27.570 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:27.565+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:27.653 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:27.647+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:27.852 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:27.846+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:27.859 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:27.855+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:28.159 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:28.154+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:28.200 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:28.196+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:28.221 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:28.215+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:28.611 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:28.606+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:28.614 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:28.610+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:28.659 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:28.655+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:28.889 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:28.884+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:28.903 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:28.898+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:29.171 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:29.166+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:29.196 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:29.192+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:29.268 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:29.263+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:29.597 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:29.591+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:29.597 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:29.593+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:29.653 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:29.648+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:29.903 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:29.898+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:29.917 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:29.912+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:30.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:30.184+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:30.224 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:30.218+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:30.288 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:30.278+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:30.610 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:30.605+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:30.613 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:30.609+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:30.703 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:30.697+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:30.938 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:30.933+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:30.963 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:30.958+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:31.205 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:31.200+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:31.234 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:31.228+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:31.327 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:31.322+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:31.639 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:31.634+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:31.644 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:31.638+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:31.696 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:31.685+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:31.958 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:31.954+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:32.013 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:32.008+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:32.162 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:32.157+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:32.212 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:32.208+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:32.377 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:32.371+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:32.599 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:32.592+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:32.600 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:32.594+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:32.737 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:32.732+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:32.955 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:32.951+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:33.025 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:33.020+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:33.196 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:33.191+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:33.207 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:33.201+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:33.364 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:33.358+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:33.568 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:33.561+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:33.576 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:33.569+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:33.736 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:33.731+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:33.985 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:33.981+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:34.062 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:34.056+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:34.165 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:34.157+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:34.218 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:34.214+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:34.388 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:34.383+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:34.616 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:34.611+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:34.617 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:34.612+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:34.723 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:34.719+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:35.039 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:35.034+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:35.083 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:35.078+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:35.200 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:35.195+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:35.214 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:35.210+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:35.382 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:35.377+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:35.651 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:35.645+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:35.652 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:35.648+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:35.744 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:35.737+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:36.034 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:36.029+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:36.106 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:36.101+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:36.204 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:36.199+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:36.229 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:36.223+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:36.379 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:36.373+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:36.640 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:36.629+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:36.642 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:36.631+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:36.719 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:36.715+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:37.049 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:37.044+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:37.082 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:37.078+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:37.225 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:37.220+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:37.251 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:37.246+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:37.387 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:37.382+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:37.665 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:37.660+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:37.666 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:37.662+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:37.674 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:37.669+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:38.063 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:38.060+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:38.099 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:38.093+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:38.203 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:38.198+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:38.270 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:38.266+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:38.395 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:38.392+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:38.682 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:38.677+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:38.683 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:38.679+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:38.693 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:38.688+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:39.093 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:39.089+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:39.126 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:39.122+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:39.235 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:39.230+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:39.326 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:39.321+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:39.362 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:39.358+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:39.658 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:39.652+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:39.659 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:39.654+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:39.729 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:39.724+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:40.068 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:40.064+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:40.094 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:40.090+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:40.237 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:40.233+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:40.284 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:40.281+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:40.381 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:40.377+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:40.633 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:40.628+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:40.634 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:40.630+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:40.733 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:40.729+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:41.052 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:41.048+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:41.063 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:41.058+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:41.271 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:41.267+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:41.278 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:41.273+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:41.408 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:41.403+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:41.587 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:41.584+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:41.590 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:41.587+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:41.709 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:41.705+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:42.030 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:42.026+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:42.069 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:42.066+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:42.241 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:42.237+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:42.305 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:42.300+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:42.386 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:42.382+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:42.584 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:42.580+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:42.587 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:42.583+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:42.716 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:42.711+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:00:43.015 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:00:43.011+0000 16949700 -1 osd.7 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:34.472368+0000 front 2019-11-25T15:59:34.655657+0000 (oldest deadline 2019-11-25T16:00:20.199702+0000) 2019-11-25T16:00:43.097 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:00:43.092+0000 16949700 -1 osd.2 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.829681+0000 front 2019-11-25T15:59:38.813865+0000 (oldest deadline 2019-11-25T16:00:23.355529+0000) 2019-11-25T16:00:43.217 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:00:43.213+0000 16949700 -1 osd.4 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:38.882272+0000 front 2019-11-25T15:59:38.866179+0000 (oldest deadline 2019-11-25T16:00:21.090957+0000) 2019-11-25T16:00:43.351 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:00:43.347+0000 16949700 -1 osd.1 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.323637+0000 front 2019-11-25T15:59:36.611145+0000 (oldest deadline 2019-11-25T16:00:19.679737+0000) 2019-11-25T16:00:43.429 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:00:43.424+0000 16949700 -1 osd.5 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:37.201629+0000 front 2019-11-25T15:59:37.186421+0000 (oldest deadline 2019-11-25T16:00:23.069450+0000) 2019-11-25T16:00:43.568 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:43.563+0000 16949700 -1 osd.3 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:36.711455+0000 front 2019-11-25T15:59:36.368199+0000 (oldest deadline 2019-11-25T16:00:21.124382+0000) 2019-11-25T16:00:43.572 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:00:43.568+0000 16949700 -1 osd.3 258 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e256) 2019-11-25T16:00:43.706 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:00:43.703+0000 16949700 -1 osd.0 258 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T15:59:35.072321+0000 front 2019-11-25T15:59:35.071748+0000 (oldest deadline 2019-11-25T16:00:20.598772+0000) 2019-11-25T16:01:38.579 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:01:38.557+0000 39278700 -1 osd.6 263 build_incremental_map_msg missing incremental map 263 2019-11-25T16:01:38.757 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:01:38.743+0000 39278700 -1 osd.6 263 build_incremental_map_msg missing incremental map 263 2019-11-25T16:01:38.786 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:01:38.774+0000 35a71700 -1 osd.6 263 build_incremental_map_msg missing incremental map 263 2019-11-25T16:01:38.863 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:01:38.813+0000 3aa7b700 -1 osd.6 263 build_incremental_map_msg missing incremental map 263 2019-11-25T16:01:38.907 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:01:38.846+0000 3b27c700 -1 osd.6 263 build_incremental_map_msg missing incremental map 263 2019-11-25T16:01:39.121 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:01:38.939+0000 37274700 -1 osd.6 263 build_incremental_map_msg missing incremental map 263 2019-11-25T16:02:34.235 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:47:38.818 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:34.250 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:47:38.856 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:34.252 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:47:38.858 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:34.254 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:47:38.861 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:34.255 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:47:38.862 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:34.259 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:47:38.842 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:34.519 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:47:39.125 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:34.527 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:47:39.133 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:34.557 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:47:39.140 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:02:39.493 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:39.486+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:39.579 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:39.573+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:40.171 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:40.162+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:40.484 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:40.475+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:40.599 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:40.594+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:41.217 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:41.212+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:41.248 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:41.244+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:41.490 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:41.484+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:41.590 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:41.583+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:42.226 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:42.220+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:42.269 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:42.263+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:42.537 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:42.529+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:42.565 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:42.560+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:43.204 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:43.198+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:43.230 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:43.226+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:43.261 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:43.257+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:43.506 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:43.501+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:43.566 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:43.560+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:43.670 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:43.663+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:43.813 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:43.809+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:44.211 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:44.204+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:44.220 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:44.213+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:44.261 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:44.255+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:44.496 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:44.491+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:44.570 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:44.559+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:44.631 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:44.625+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:44.840 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:44.834+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:45.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:45.191+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:45.246 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:45.241+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:45.289 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:45.284+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:45.534 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:45.528+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:45.592 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:45.586+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:45.594 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:45.588+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:45.807 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:45.801+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:46.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:46.192+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:46.269 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:46.264+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:46.288 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:46.284+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:46.556 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:46.550+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:46.586 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:46.581+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:46.599 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:46.594+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:46.786 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:46.781+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:47.241 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:47.235+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:47.260 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:47.256+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:47.294 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:47.288+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:47.587 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:47.580+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:47.623 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:47.618+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:47.638 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:47.632+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:47.792 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:47.787+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:48.214 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:48.208+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:48.282 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:48.278+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:48.285 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:48.281+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:48.615 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:48.609+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:48.618 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:48.612+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:48.633 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:48.627+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:48.811 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:48.806+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:49.224 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:49.218+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:49.300 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:49.295+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:49.334 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:49.330+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:49.636 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:49.630+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:49.642 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:49.635+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:49.642 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:49.637+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:49.861 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:49.857+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:50.229 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:50.223+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:50.306 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:50.301+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:50.319 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:50.315+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:50.609 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:50.604+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:50.620 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:50.615+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:50.697 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:50.691+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:50.920 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:50.913+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:51.261 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:51.257+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:51.268 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:51.263+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:51.294 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:51.290+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:51.569 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:51.562+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:51.639 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:51.632+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:51.686 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:51.680+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:51.945 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:51.941+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:52.227 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:52.221+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:52.244 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:52.239+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:52.258 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:52.254+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:52.595 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:52.590+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:52.662 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:52.656+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:52.690 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:52.685+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:52.924 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:52.919+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:53.211 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:53.206+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:53.224 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:53.219+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:53.309 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:53.304+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:53.604 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:53.599+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:53.705 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:53.699+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:53.745 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:53.739+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:53.895 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:53.891+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:54.163 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:54.160+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:54.266 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:54.260+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:54.289 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:54.285+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:54.584 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:54.578+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:54.705 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:54.699+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:54.736 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:54.730+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:54.861 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:54.857+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:55.122 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:55.119+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:55.224 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:55.218+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:55.308 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:55.303+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:55.565 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:55.557+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:55.720 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:55.713+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:55.722 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:55.715+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:55.848 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:55.844+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:56.139 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:56.136+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:56.185 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:56.179+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:56.352 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:56.348+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:56.573 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:56.567+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:56.752 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:56.747+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:56.771 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:56.763+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:56.805 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:56.801+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:57.141 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:57.136+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:57.229 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:57.223+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:57.312 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:57.307+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:57.608 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:57.602+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:57.709 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:57.705+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:57.742 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:57.737+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:57.769 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:57.765+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:58.120 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:58.114+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:58.246 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:58.241+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:58.333 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:58.328+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:58.661 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:58.655+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:58.666 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:58.662+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:58.732 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:58.726+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:58.814 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:58.809+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:02:59.088 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:02:59.083+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:02:59.299 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:02:59.293+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:02:59.327 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:02:59.323+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:02:59.641 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:02:59.634+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:02:59.740 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:02:59.735+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:02:59.746 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:02:59.740+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:02:59.842 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:02:59.837+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:00.042 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:00.038+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:00.298 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:00.294+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:00.343 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:00.337+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:00.681 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:00.675+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:00.738 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:00.732+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:00.748 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:00.743+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:00.799 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:00.795+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:01.075 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:01.071+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:01.298 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:01.292+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:01.338 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:01.334+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:01.645 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:01.640+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:01.733 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:01.727+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:01.775 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:01.770+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:01.807 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:01.801+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:02.051 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:02.048+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:02.313 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:02.307+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:02.381 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:02.376+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:02.631 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:02.625+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:02.757 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:02.751+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:02.758 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:02.753+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:02.796 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:02.790+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:03.007 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:03.002+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:03.345 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:03.339+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:03.370 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:03.366+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:03.599 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:03.592+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:03.716 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:03.711+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:03.790 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:03.785+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:03.797 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:03.791+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:03.988 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:03.983+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:04.377 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:04.372+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:04.399 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:04.393+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:04.615 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:04.609+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:04.707 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:04.701+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:04.779 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:04.773+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:04.803 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:04.797+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:04.969 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:04.963+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:05.344 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:05.338+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:05.400 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:05.394+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:05.609 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:05.605+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:05.726 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:05.721+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:05.807 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:05.801+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:05.817 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:05.811+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:06.002 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:05.996+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:06.312 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:06.306+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:06.418 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:06.412+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:06.593 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:06.587+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:06.734 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:06.729+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:06.810 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:06.804+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:06.866 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:06.860+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:06.992 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:06.985+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:07.313 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:07.307+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:07.421 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:07.415+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:07.625 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:07.620+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:07.748 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:07.742+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:07.844 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:07.838+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:07.893 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:07.887+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:08.012 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:08.006+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:08.302 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:08.296+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:08.455 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:08.449+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:08.609 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:08.604+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:08.767 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:08.762+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:08.852 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:08.846+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:08.892 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:08.885+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:09.044 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:09.038+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:09.329 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:09.323+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:09.506 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:09.501+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:09.632 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:09.627+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:09.753 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:09.747+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:09.898 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:09.892+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:09.927 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:09.921+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:10.014 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:10.008+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:10.295 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:10.290+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:10.483 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:10.478+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:10.598 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:10.593+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:10.805 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:10.799+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:10.856 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:10.851+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:10.923 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:10.918+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:11.037 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:11.032+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:11.298 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:11.294+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:11.520 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:11.515+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:11.645 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:11.640+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:11.857 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:11.852+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:11.876 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:11.871+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:11.914 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:11.910+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:12.030 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:12.026+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:12.323 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:12.318+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:12.566 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:12.562+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:12.657 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:12.653+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:12.812 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:12.808+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:12.891 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:12.886+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:12.964 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:12.960+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:13.054 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:13.050+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:13.353 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:13.349+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:13.568 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:13.563+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:13.697 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:13.692+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:13.863 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:13.859+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:13.902 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:13.898+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:14.012 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:14.008+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:14.066 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:14.061+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:14.115 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:03:14.111+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 41sec e264 v264) 2019-11-25T16:03:14.369 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:14.365+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:14.528 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:14.522+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:14.739 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:14.733+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:14.853 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:14.849+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:14.879 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:14.875+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:15.021 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:15.016+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:15.057 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:15.052+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:15.360 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:15.356+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:15.529 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:15.524+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:15.757 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:15.752+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:15.819 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:15.815+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:15.841 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:15.837+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:15.986 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:15.981+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:16.069 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:16.065+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:16.358 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:16.353+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:16.532 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:16.527+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:16.722 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:16.717+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:16.780 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:16.775+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:16.824 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:16.820+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:17.020 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:17.014+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:17.119 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:17.114+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:17.338 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:17.333+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:17.572 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:17.567+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:17.697 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:17.692+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:17.812 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:17.808+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:17.834 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:17.829+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:18.002 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:17.998+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:18.139 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:18.134+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:18.382 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:18.377+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:18.577 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:03:18.572+0000 16949700 -1 osd.4 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:55.223891+0000 front 2019-11-25T16:01:55.378792+0000 (oldest deadline 2019-11-25T16:02:39.212596+0000) 2019-11-25T16:03:18.683 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:03:18.679+0000 16949700 -1 osd.6 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.868328+0000 front 2019-11-25T16:01:54.869631+0000 (oldest deadline 2019-11-25T16:02:38.869887+0000) 2019-11-25T16:03:18.804 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:03:18.799+0000 16949700 -1 osd.1 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.294431+0000 front 2019-11-25T16:01:58.428740+0000 (oldest deadline 2019-11-25T16:02:43.571287+0000) 2019-11-25T16:03:18.830 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:03:18.825+0000 16949700 -1 osd.7 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.040008+0000 front 2019-11-25T16:01:58.048674+0000 (oldest deadline 2019-11-25T16:02:43.163835+0000) 2019-11-25T16:03:19.039 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:03:19.034+0000 16949700 -1 osd.2 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:58.003744+0000 front 2019-11-25T16:01:57.365156+0000 (oldest deadline 2019-11-25T16:02:42.366396+0000) 2019-11-25T16:03:19.135 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:03:19.131+0000 16949700 -1 osd.5 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:57.550821+0000 front 2019-11-25T16:01:57.583759+0000 (oldest deadline 2019-11-25T16:02:39.052444+0000) 2019-11-25T16:03:19.200 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:03:19.194+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 41sec e264 v264) 2019-11-25T16:03:19.353 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:03:19.350+0000 16949700 -1 osd.0 264 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:01:54.526566+0000 front 2019-11-25T16:01:54.881118+0000 (oldest deadline 2019-11-25T16:02:40.390947+0000) 2019-11-25T16:03:51.542 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:03:51.532+0000 35270700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-11-25T16:03:51.582 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:03:51.571+0000 3b27c700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-11-25T16:03:51.621 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:03:51.587+0000 3aa7b700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-11-25T16:03:51.764 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:03:51.748+0000 33a6d700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-11-25T16:03:51.800 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:03:51.779+0000 3a27a700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-11-25T16:03:51.804 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:03:51.798+0000 3b27c700 -1 osd.3 269 build_incremental_map_msg missing incremental map 269 2019-11-25T16:04:02.002 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:01.984+0000 3b27c700 -1 osd.3 270 build_incremental_map_msg missing incremental map 269 2019-11-25T16:04:16.960 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:16.953+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:17.263 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:17.259+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:17.605 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:17.600+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:17.931 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:17.928+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:18.274 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:18.270+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:18.640 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:18.636+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:18.984 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:18.980+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:19.297 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:19.292+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:19.614 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:19.609+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:20.037 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:20.033+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:20.326 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:20.322+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:20.620 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:20.615+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:21.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:21.030+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:21.377 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:21.372+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:21.599 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:21.593+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:22.020 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:22.016+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:22.071 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:22.068+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:22.266 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:22.261+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:22.395 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:22.391+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:22.444 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:22.438+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:22.563 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:22.559+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:23.059 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:23.054+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:23.070 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:23.067+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:23.225 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:23.220+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:23.396 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:23.391+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:23.431 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:23.426+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:23.570 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:23.566+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:24.059 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:24.056+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:24.098 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:24.094+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:24.262 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:24.257+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:24.352 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:24.348+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:24.393 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:24.389+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:24.539 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:24.535+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:25.055 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:25.050+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:25.062 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:25.058+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:25.311 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:25.307+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:25.361 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:25.356+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:25.441 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:25.437+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:25.505 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:25.500+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:26.052 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:26.048+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:26.079 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:26.074+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:26.360 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:26.354+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:26.413 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:26.408+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:26.468 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:26.464+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:26.539 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:26.534+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:27.032 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:27.027+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:27.033 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:27.029+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:27.043 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:27.039+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:27.368 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:27.362+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:27.437 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:27.433+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:27.464 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:27.459+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:27.571 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:27.566+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:27.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:27.988+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:27.997 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:27.993+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:28.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:28.030+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:28.334 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:28.329+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:28.400 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:28.395+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:28.493 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:28.489+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:28.539 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:28.535+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:28.978 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:28.971+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:28.979 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:28.974+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:29.083 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:29.079+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:29.374 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:29.368+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:29.389 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:29.385+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:29.504 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:29.499+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:29.547 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:29.542+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:30.004 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:29.999+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:30.005 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:30.001+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:30.076 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:30.071+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:30.369 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:30.364+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:30.427 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:30.421+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:30.509 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:30.503+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:30.514 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:30.510+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:31.042 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:31.037+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:31.047 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:31.042+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:31.047 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:31.044+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:31.324 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:31.320+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:31.441 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:31.435+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:31.534 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:31.529+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:31.557 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:31.552+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:32.020 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:32.015+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:32.048 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:32.043+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:32.048 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:32.044+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:32.335 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:32.330+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:32.423 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:32.417+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:32.498 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:32.493+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:32.527 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:32.523+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:33.013 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:33.009+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:33.045 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:33.040+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:33.046 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:33.042+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:33.324 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:33.319+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:33.421 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:33.414+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:33.467 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:33.462+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:33.533 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:33.528+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:34.049 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:34.044+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:34.050 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:34.046+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:34.057 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:34.053+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:34.280 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:34.276+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:34.448 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:34.443+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:34.483 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:34.478+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:34.569 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:34.563+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:35.044 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:35.040+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:35.047 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:35.044+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:35.062 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:35.056+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:35.308 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:35.303+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:35.426 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:35.421+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:35.533 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:35.528+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:35.543 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:35.539+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:35.647 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T16:04:35.568+0000 20e21700 -1 RGWWatcher::handle_error cookie 384607280 err (107) Transport endpoint is not connected 2019-11-25T16:04:36.031 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:36.027+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:36.035 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:36.030+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:36.036 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:36.032+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:36.334 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:36.329+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:36.423 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:36.418+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:36.426 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:36.422+0000 16949700 -1 osd.4 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:36.500 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:36.495+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:36.524 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:36.519+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:37.059 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:37.054+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:37.086 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:37.079+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:37.087 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:37.080+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:37.354 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:37.350+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:37.406 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:37.400+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:37.407 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:37.402+0000 16949700 -1 osd.4 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:37.468 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:37.464+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:37.530 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:37.525+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:38.078 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:38.073+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:38.112 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:38.109+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:38.114 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:38.111+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:38.358 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:38.353+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:38.377 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:38.371+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:38.377 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:38.372+0000 16949700 -1 osd.4 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:38.428 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:38.422+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:38.506 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:38.501+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:39.036 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:39.031+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:39.092 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:39.089+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:39.094 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:39.091+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:39.365 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:39.361+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:39.367 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:39.361+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:39.368 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:39.363+0000 16949700 -1 osd.4 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:39.459 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:39.454+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:39.483 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:39.479+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:40.008 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:40.004+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:40.145 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:40.140+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:40.146 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:40.142+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:40.339 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:40.335+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:40.410 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:40.404+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:40.411 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:40.406+0000 16949700 -1 osd.4 270 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:40.472 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:40.468+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:40.534 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:40.528+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:41.037 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:41.032+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:41.174 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:41.169+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:41.174 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:41.171+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:41.316 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:41.312+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:41.394 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:41.389+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:41.395 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:41.390+0000 16949700 -1 osd.4 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:41.439 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:41.435+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:41.505 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:41.500+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:42.048 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:42.044+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:42.135 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:42.128+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:42.136 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:42.131+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:42.291 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:42.286+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:42.403 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:42.399+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:42.407 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:42.402+0000 16949700 -1 osd.4 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:42.445 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:42.440+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:42.571 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:42.559+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:43.086 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:43.082+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:43.127 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:43.122+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:43.128 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:43.124+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:43.309 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:43.304+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:43.402 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:43.396+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:43.402 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:43.398+0000 16949700 -1 osd.4 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:43.423 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:43.418+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:43.565 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:43.560+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:44.111 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:44.106+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:44.112 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:44.108+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:44.132 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:44.128+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:44.338 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:44.334+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:44.392 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:44.387+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:44.431 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:44.426+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:44.433 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:44.429+0000 16949700 -1 osd.4 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:44.558 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:44.554+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:45.075 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:45.069+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:45.076 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:45.071+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:45.137 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:45.133+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:45.339 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:45.334+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:45.350 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:45.345+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:45.418 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:45.412+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:45.419 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:45.414+0000 16949700 -1 osd.4 270 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:45.613 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:45.607+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:46.103 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:46.097+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:46.105 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:46.099+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:46.123 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:46.119+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:46.292 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:46.288+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:46.338 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:46.333+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:46.450 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:46.444+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:46.451 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:46.446+0000 16949700 -1 osd.4 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:46.576 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:46.571+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:47.145 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:47.141+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:47.149 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:47.145+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:47.184 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:47.180+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:47.280 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:47.276+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:47.333 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:47.327+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:47.498 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:47.492+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:47.499 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:47.494+0000 16949700 -1 osd.4 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:47.616 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:47.612+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:48.149 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:48.145+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:48.197 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:48.193+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:48.201 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:48.197+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:48.238 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:48.234+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:48.329 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:48.323+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:48.553 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:48.547+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:48.553 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:48.549+0000 16949700 -1 osd.4 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:48.621 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:48.616+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:49.193 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:49.188+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:49.202 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:49.197+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:49.202 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:49.198+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:49.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:49.199+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:49.300 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:49.294+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:49.475 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:04:49.468+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e270 v270) 2019-11-25T16:04:49.515 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:49.511+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:49.518 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:49.514+0000 16949700 -1 osd.4 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:49.590 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:49.586+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:50.189 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:50.183+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:50.216 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:50.211+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:50.243 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:50.238+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:50.247 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:50.242+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:50.282 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:50.277+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:50.572 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:50.559+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:50.580 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:50.572+0000 16949700 -1 osd.4 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:50.596 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:50.590+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:51.175 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:04:51.170+0000 16949700 -1 osd.2 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:36.926498+0000 front 2019-11-25T16:03:36.935922+0000 (oldest deadline 2019-11-25T16:04:21.921238+0000) 2019-11-25T16:04:51.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:51.197+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:51.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:51.198+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:51.210 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:51.205+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:51.288 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:51.284+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:51.571 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:51.566+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:51.572 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:51.568+0000 16949700 -1 osd.4 270 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:51.642 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:51.636+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:52.172 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:52.167+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:52.222 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:52.217+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:52.224 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:52.218+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:52.261 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:52.257+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:52.600 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:52.596+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:52.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:52.600+0000 16949700 -1 osd.4 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:52.621 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:52.614+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:53.176 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:53.171+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:53.242 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:53.238+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:53.263 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:53.257+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:53.264 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:53.259+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:53.607 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:53.604+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:53.610 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:53.607+0000 16949700 -1 osd.4 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:53.658 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:53.654+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:54.146 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:54.140+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:54.216 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:54.212+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:54.296 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:54.291+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:54.300 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:54.295+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:54.498 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:04:54.493+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e270 v270) 2019-11-25T16:04:54.634 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:54.631+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:54.646 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:54.642+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:54.646 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:54.643+0000 16949700 -1 osd.4 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:55.149 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:55.144+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:55.178 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:55.174+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:55.329 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:55.324+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:55.329 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:55.325+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:55.637 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:55.632+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:55.638 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:55.633+0000 16949700 -1 osd.4 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:55.654 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:55.651+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:56.159 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:56.154+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:56.160 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:56.156+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:56.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:56.348+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:56.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:56.350+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:56.613 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:56.609+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:56.616 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:56.613+0000 16949700 -1 osd.4 270 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:56.641 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:56.636+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:57.137 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:57.133+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:57.187 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:57.183+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:57.409 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:57.403+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:57.410 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:57.405+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:57.625 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:57.620+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:57.626 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:57.622+0000 16949700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:57.682 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:57.678+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:58.164 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:58.160+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:58.189 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:58.185+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:58.415 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:58.409+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:58.416 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:58.410+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:58.605 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:58.601+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:58.608 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:58.605+0000 16949700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:58.717 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:58.712+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:04:59.193 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:04:59.189+0000 16949700 -1 osd.0 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.065515+0000 front 2019-11-25T16:03:34.283408+0000 (oldest deadline 2019-11-25T16:04:16.600867+0000) 2019-11-25T16:04:59.205 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:04:59.201+0000 16949700 -1 osd.5 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:33.231405+0000 front 2019-11-25T16:03:34.263108+0000 (oldest deadline 2019-11-25T16:04:17.004634+0000) 2019-11-25T16:04:59.391 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:59.386+0000 16949700 -1 osd.3 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:03:46.928670+0000 (oldest deadline 2019-11-25T16:04:26.928670+0000) 2019-11-25T16:04:59.394 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:04:59.390+0000 16949700 -1 osd.3 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e268) 2019-11-25T16:04:59.579 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:04:59.569+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e270 v270) 2019-11-25T16:04:59.640 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:59.633+0000 16949700 -1 osd.4 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.321938+0000 front 2019-11-25T16:03:37.012335+0000 (oldest deadline 2019-11-25T16:04:21.381733+0000) 2019-11-25T16:04:59.641 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:04:59.634+0000 16949700 -1 osd.4 270 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:04:59.698 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:04:59.695+0000 16949700 -1 osd.7 270 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:03:37.272501+0000 front 2019-11-25T16:03:37.321081+0000 (oldest deadline 2019-11-25T16:04:17.177158+0000) 2019-11-25T16:05:00.669 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:00.665+0000 16949700 -1 osd.4 271 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:01.659 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:01.657+0000 16949700 -1 osd.4 272 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:02.710 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:02.706+0000 16949700 -1 osd.4 272 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:03.715 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:03.711+0000 16949700 -1 osd.4 272 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:04.703 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:04.698+0000 16949700 -1 osd.4 272 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:05.743 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:05.738+0000 16949700 -1 osd.4 272 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:06.792 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:06.786+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:07.790 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:07.785+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:08.842 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:08.837+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:09.878 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:09.874+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:10.913 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:10.908+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:11.931 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:11.926+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:12.962 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:12.957+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:13.993 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:13.989+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:14.957 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:14.952+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:15.930 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:15.926+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:16.972 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:16.967+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:17.963 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:17.958+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:18.953 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:18.949+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:19.986 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:19.982+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:21.033 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:21.028+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:22.080 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:22.076+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:23.085 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:23.080+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:24.085 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:24.079+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:25.050 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:25.044+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:26.098 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:26.094+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:27.105 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:27.100+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:28.094 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:28.090+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:29.099 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:29.093+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:30.102 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:30.097+0000 16949700 -1 osd.4 272 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113508 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:31.331 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:05:31.256+0000 16949700 -1 osd.4 273 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:113516 5.3 5:c609908c:::notify.5:head [watch ping cookie 384607280 gen 5] snapc 0=[] ondisk+write+known_if_redirected e270) 2019-11-25T16:05:50.184 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:50:54.767 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:05:50.186 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:50:54.769 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:05:50.347 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:50:54.953 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:05:50.387 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:50:54.994 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:05:50.391 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:50:54.997 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:05:50.391 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:50:54.998 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:05:50.446 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:50:55.030 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:05:50.627 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:50:55.233 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:05:50.675 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:50:55.281 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.423 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:51:05.006 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.464 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:51:05.070 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.490 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:51:05.073 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.491 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:51:05.098 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.511 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:51:05.117 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.519 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:51:05.126 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.528 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:51:05.135 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.544 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:51:05.127 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:00.549 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:51:05.155 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.631 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:51:31.214 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.645 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:51:31.228 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.667 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:51:31.273 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.673 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:51:31.279 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.679 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:51:31.286 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.683 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:51:31.290 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.724 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:51:31.330 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.737 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:51:31.344 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:26.752 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:51:31.335 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:06:56.779 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:06:56.774+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:06:56.864 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:06:56.856+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:06:56.915 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:06:56.910+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:06:56.970 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:06:56.965+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:06:57.830 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:06:57.824+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:06:57.901 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:06:57.896+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:06:57.906 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:06:57.900+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:06:57.969 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:06:57.965+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:06:57.999 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:06:57.995+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:06:58.701 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:06:58.696+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:06:58.817 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:06:58.810+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:06:58.869 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:06:58.862+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:06:58.937 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:06:58.931+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:06:58.953 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:06:58.947+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:06:59.019 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:06:59.014+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:06:59.502 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:06:59.497+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:06:59.670 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:06:59.667+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:06:59.778 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:06:59.772+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:06:59.910 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:06:59.904+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:06:59.920 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:06:59.915+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:06:59.937 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:06:59.932+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:06:59.980 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:06:59.973+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:00.517 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:00.512+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:00.666 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:00.662+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:00.768 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:00.762+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:00.901 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:00.893+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:00.911 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:00.907+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:00.934 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:00.929+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:00.943 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:00.938+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:01.547 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:01.540+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:01.685 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:01.681+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:01.743 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:01.737+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:01.887 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:01.881+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:01.905 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:01.899+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:01.939 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:01.934+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:01.943 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:01.937+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:02.530 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:02.523+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:02.734 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:02.729+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:02.775 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:02.769+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:02.908 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:02.901+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:02.926 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:02.922+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:02.930 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:02.924+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:02.968 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:02.962+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:03.564 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:03.558+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:03.758 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:03.752+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:03.776 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:03.772+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:03.925 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:03.919+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:03.929 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:03.923+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:03.935 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:03.929+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:03.957 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:03.953+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:04.576 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:04.570+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:04.739 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:04.734+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:04.760 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:04.754+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:04.916 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:04.910+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:04.928 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:04.922+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:04.953 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:04.949+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:04.958 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:04.953+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:05.608 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:05.603+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:05.755 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:05.750+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:05.775 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:05.770+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:05.924 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:05.918+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:05.926 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:05.920+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:05.932 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:05.927+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:05.989 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:05.984+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:06.578 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:06.572+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:06.715 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:06.709+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:06.772 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:06.767+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:06.885 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:06.878+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:06.916 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:06.909+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:06.972 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:06.967+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:07.004 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:07.000+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:07.534 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:07.528+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:07.729 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:07.723+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:07.735 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:07.732+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:07.861 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:07.855+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:07.873 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:07.867+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:07.983 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:07.979+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:08.043 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:08.037+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:08.490 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:08.484+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:08.701 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:08.697+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:08.736 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:08.730+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:08.829 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:08.822+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:08.856 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:08.850+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:08.951 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:08.947+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:09.048 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:09.043+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:09.513 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:09.506+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:09.741 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:09.735+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:09.757 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:09.751+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:09.828 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:09.822+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:09.829 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:09.823+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:09.935 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:09.930+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:10.063 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:10.059+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:10.506 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:10.499+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:10.711 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:10.706+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:10.782 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:10.776+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:10.862 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:10.856+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:10.865 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:10.859+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:10.905 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:10.899+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:11.049 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:11.043+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:11.485 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:11.480+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:11.721 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:11.717+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:11.835 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:11.829+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:11.843 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:11.837+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:11.873 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:11.867+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:11.896 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:11.891+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:12.080 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:12.076+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:12.532 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:12.527+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:12.760 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:12.755+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:12.808 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:12.802+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:12.869 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:12.863+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:12.872 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:12.866+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:12.882 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:12.878+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:13.090 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:13.085+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:13.510 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:13.504+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:13.777 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:13.773+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:13.805 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:13.799+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:13.834 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:13.827+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:13.873 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:13.868+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:13.932 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:13.928+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:14.128 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:14.122+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:14.522 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:14.515+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:14.732 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:14.728+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:14.835 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:14.829+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:14.836 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:14.831+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:14.854 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:14.848+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:14.950 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:14.945+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:15.093 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:15.088+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:15.567 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:15.562+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:15.769 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:15.764+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:15.819 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:15.813+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:15.852 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:15.846+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:15.863 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:15.857+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:15.971 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:15.966+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:16.142 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:16.135+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:16.594 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:16.587+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:16.740 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:16.736+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:16.830 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:16.825+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:16.886 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:16.881+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:16.896 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:16.891+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:16.993 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:16.988+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:17.176 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:17.171+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:17.562 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:17.554+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:17.716 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:17.710+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:17.831 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:17.825+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:17.887 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:17.881+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:17.896 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:17.889+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:17.993 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:17.988+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:18.220 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:18.216+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:18.519 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:18.513+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:18.736 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:18.731+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:18.861 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:18.855+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:18.885 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:18.880+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:18.936 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:18.926+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:18.999 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:18.994+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:19.244 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:19.239+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:19.541 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:19.535+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:19.777 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:19.773+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:19.857 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:19.851+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:19.889 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:19.883+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:19.925 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:19.919+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:20.018 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:20.011+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:20.217 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:20.213+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:20.519 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:20.514+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:20.770 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:20.766+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:20.902 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:20.896+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:20.920 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:20.914+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:20.930 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:20.925+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:20.988 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:20.983+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:21.233 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:21.228+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:21.521 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:21.515+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:21.738 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:21.732+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:21.897 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:21.891+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:21.935 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:21.929+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:21.949 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:21.944+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:21.986 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:21.980+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:22.207 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:22.202+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:22.561 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:22.556+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:22.790 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:22.784+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:22.889 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:22.884+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:22.941 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:22.934+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:22.967 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:22.962+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:22.991 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:22.982+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:23.173 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:23.168+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:23.519 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:23.514+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:23.766 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:23.761+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:23.926 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:23.921+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:23.947 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:23.942+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:23.966 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:23.961+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:23.979 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:23.975+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:24.223 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:24.219+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:24.566 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:24.561+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:24.805 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:24.801+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:24.936 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:24.932+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:24.946 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:24.941+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:24.970 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:24.966+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:24.981 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:24.977+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:25.269 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:25.264+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:25.618 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:25.613+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:25.835 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:25.830+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:25.967 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:25.960+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:25.970 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:25.965+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:25.990 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:25.986+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:25.990 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:25.985+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:26.317 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:26.311+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:26.610 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:26.606+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:26.822 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:26.818+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:26.959 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:26.954+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:26.964 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:26.959+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:26.992 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:26.987+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:26.994 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:26.989+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:27.278 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:27.272+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:27.574 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:27.569+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:27.804 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:27.799+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:27.933 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:27.928+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:27.957 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:27.951+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:27.958 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:27.953+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:27.995 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:27.991+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:28.280 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:28.275+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:28.536 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:28.532+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:28.792 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:28.786+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:28.892 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:28.887+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:28.930 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:28.925+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:28.946 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:28.941+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:29.001 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:28.997+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:29.289 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:29.284+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:29.560 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:29.555+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:29.824 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:29.818+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:29.900 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:29.896+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:29.929 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:29.921+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:29.990 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:29.984+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:30.018 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:30.014+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:30.255 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:30.249+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:30.570 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:30.565+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:30.596 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:07:30.593+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 43sec e276 v276) 2019-11-25T16:07:30.834 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:30.828+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:30.923 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:30.917+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:30.932 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:30.927+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:31.010 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:31.005+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:31.031 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:31.025+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:31.232 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:31.226+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:31.622 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:31.617+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:31.876 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:31.870+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:31.891 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:31.887+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:31.965 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:31.960+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:32.003 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:31.998+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:32.009 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:32.002+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:32.247 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:32.242+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:32.626 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:32.621+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:32.877 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:32.872+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:32.899 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:32.894+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:32.979 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:32.975+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:33.038 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:33.030+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:33.049 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:33.044+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:33.232 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:33.228+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:33.599 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:33.594+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:33.931 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:33.926+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:33.938 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:33.934+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:33.999 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:33.994+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:34.039 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:34.035+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:34.049 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:34.043+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:34.187 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:34.182+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:34.600 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:34.596+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:34.898 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:07:34.893+0000 16949700 -1 osd.7 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.691127+0000 front 2019-11-25T16:06:15.952097+0000 (oldest deadline 2019-11-25T16:06:55.900079+0000) 2019-11-25T16:07:34.961 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:34.955+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:34.981 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:07:34.977+0000 16949700 -1 osd.4 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:14.787431+0000 front 2019-11-25T16:06:14.729144+0000 (oldest deadline 2019-11-25T16:06:57.573644+0000) 2019-11-25T16:07:35.037 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:07:35.032+0000 16949700 -1 osd.5 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:15.676290+0000 front 2019-11-25T16:06:16.423527+0000 (oldest deadline 2019-11-25T16:06:56.685945+0000) 2019-11-25T16:07:35.092 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:35.088+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:07:35.210 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:07:35.206+0000 16949700 -1 osd.2 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:11.318707+0000 front 2019-11-25T16:06:11.169509+0000 (oldest deadline 2019-11-25T16:06:56.441574+0000) 2019-11-25T16:07:35.561 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:07:35.556+0000 16949700 -1 osd.6 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.767043+0000 front 2019-11-25T16:06:14.091142+0000 (oldest deadline 2019-11-25T16:06:59.012198+0000) 2019-11-25T16:07:35.680 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:07:35.675+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 43sec e276 v276) 2019-11-25T16:07:35.932 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:07:35.928+0000 16949700 -1 osd.0 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:13.430495+0000 front 2019-11-25T16:06:13.131578+0000 (oldest deadline 2019-11-25T16:06:58.124546+0000) 2019-11-25T16:07:36.093 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:07:36.088+0000 16949700 -1 osd.1 276 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:06:12.026662+0000 front 2019-11-25T16:06:12.021196+0000 (oldest deadline 2019-11-25T16:06:56.686075+0000) 2019-11-25T16:08:12.106 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:11.847+0000 33e6e700 -1 osd.0 280 build_incremental_map_msg missing incremental map 280 2019-11-25T16:08:14.831 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:14.731+0000 37a75700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:14.918 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:14.793+0000 124aa700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:14.939 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:14.796+0000 33a6d700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:14.949 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:14.803+0000 38a77700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:15.051 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:14.956+0000 34a6f700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:15.076 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:15.038+0000 3b27c700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:15.084 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:15.040+0000 34a6f700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:15.180 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:15.073+0000 38a77700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:22.551 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:22.537+0000 3aa7b700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:23.846 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:23.835+0000 37274700 -1 osd.3 282 build_incremental_map_msg missing incremental map 282 2019-11-25T16:08:33.871 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:33.866+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:34.906 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:34.900+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:35.578 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:35.573+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:35.864 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:35.858+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:36.206 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:36.200+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:36.324 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:36.319+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:36.534 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:36.530+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:36.615 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:36.611+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:36.859 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:36.854+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:37.168 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:37.163+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:37.169 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:37.164+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:37.352 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:37.347+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:37.495 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:37.491+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:37.574 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:37.569+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:37.830 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:37.825+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:38.131 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:38.127+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:38.171 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:38.166+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:38.351 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:38.342+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:38.532 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:38.527+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:38.628 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:38.624+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:38.830 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:38.825+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:39.142 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:39.138+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:39.218 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:39.213+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:39.325 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:39.320+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:39.550 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:39.546+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:39.679 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:39.674+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:39.875 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:39.870+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:40.186 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:40.181+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:40.193 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:40.188+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:40.380 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:40.375+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:40.550 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:40.542+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:40.655 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:40.650+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:40.911 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:40.906+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:41.179 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:41.173+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:41.182 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:41.177+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:41.428 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:41.424+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:41.597 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:41.592+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:41.634 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:41.630+0000 16949700 -1 osd.3 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:41.678 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:41.672+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:41.907 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:41.902+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:42.180 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:42.175+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:42.212 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:42.207+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:42.467 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:42.462+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:42.623 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:42.618+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:42.641 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:42.637+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:42.699 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:42.694+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:42.905 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:42.900+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:43.224 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:43.220+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:43.267 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:43.261+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:43.442 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:43.436+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:43.674 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:43.669+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:43.678 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:43.674+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:43.733 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:43.726+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:43.952 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:43.947+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:44.225 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:44.220+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:44.316 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:44.311+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:44.415 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:44.411+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:44.664 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:44.659+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:44.693 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:44.688+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:44.746 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:44.741+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:45.006 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:45.001+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:45.204 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:45.199+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:45.325 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:45.320+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:45.401 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:45.396+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:45.686 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:45.681+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:45.708 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:45.704+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:45.716 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:45.711+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:45.988 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:45.983+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:46.183 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:46.179+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:46.313 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:46.307+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:46.386 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:46.381+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:46.684 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:46.678+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:46.688 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:46.683+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:46.753 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:46.749+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:47.001 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:46.996+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:47.199 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:47.195+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:47.314 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:47.308+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:47.419 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:47.414+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:47.648 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:47.643+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:47.682 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:47.677+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:47.771 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:47.767+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:47.963 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:47.958+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:48.237 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:48.233+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:48.366 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:48.361+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:48.384 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:48.376+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:48.652 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:48.647+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:48.652 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:48.648+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:48.790 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:48.787+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:49.001 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:48.995+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:49.248 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:49.244+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:49.345 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:49.339+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:49.370 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:49.365+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:49.636 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:49.631+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:49.695 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:49.690+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:49.802 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:49.799+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:49.805 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:49.802+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:49.987 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:49.982+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:50.232 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:50.227+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:50.346 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:50.340+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:50.390 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:50.385+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:50.682 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:50.678+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:50.736 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:50.728+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:50.776 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:50.771+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:50.778 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:50.774+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:50.945 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:50.940+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:51.236 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:51.232+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:51.359 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:51.354+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:51.394 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:51.389+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:51.696 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:51.692+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:51.709 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:51.704+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:51.774 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:51.769+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:51.775 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:51.772+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:51.999 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:51.994+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:52.244 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:52.240+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:52.351 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:52.345+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:52.431 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:52.421+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:52.694 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:52.690+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:52.737 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:52.732+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:52.738 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:52.734+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:52.747 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:52.742+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:53.042 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:53.037+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:53.241 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:53.236+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:53.345 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:53.339+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:53.436 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:53.431+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:53.719 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:53.713+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:53.721 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:53.714+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:53.735 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:53.730+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:53.788 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:53.777+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:54.021 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:54.016+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:54.269 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:54.264+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:54.377 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:54.372+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:54.473 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:54.468+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:54.696 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:54.692+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:54.698 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:54.693+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:54.698 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:54.695+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:54.746 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:54.741+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:55.009 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:55.003+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:55.269 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:55.264+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:55.416 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:55.410+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:55.473 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:55.468+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:55.678 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:55.674+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:55.679 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:55.675+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:55.715 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:55.712+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:55.729 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:55.726+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:56.038 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:56.033+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:56.314 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:56.308+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:56.420 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:56.415+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:56.493 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:56.488+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:56.684 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:56.678+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:56.684 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:56.680+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:56.716 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:56.712+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:56.761 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:56.756+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:57.009 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:57.003+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:57.275 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:57.270+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:57.407 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:57.400+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:57.480 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:57.475+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:57.674 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:57.670+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:57.723 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:57.719+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:57.728 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:57.722+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:57.807 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:57.803+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:58.008 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:58.002+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:58.288 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:58.283+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:58.455 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:58.450+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:58.535 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:58.530+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:58.639 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:58.634+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:58.738 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:58.734+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:58.741 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:58.738+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:58.847 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:58.843+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:08:59.037 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:08:59.032+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:08:59.334 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:08:59.329+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:08:59.441 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:08:59.435+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:08:59.490 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:08:59.484+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:08:59.603 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:08:59.598+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:08:59.729 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:59.724+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:08:59.730 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:08:59.726+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:08:59.818 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:08:59.813+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:00.039 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:00.033+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:00.358 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:00.354+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:00.421 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:00.415+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:00.462 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:00.457+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:00.656 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:00.652+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:00.779 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:00.764+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:00.781 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:00.766+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:00.870 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:00.866+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:01.026 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:01.022+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:01.395 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:01.391+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:01.430 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:01.424+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:01.488 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:01.483+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:01.678 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:01.673+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:01.723 INFO:tasks.rgw.client.0.smithi153.stdout:2019-11-25T16:09:01.437+0000 20e21700 -1 RGWWatcher::handle_error cookie 5442140224 err (107) Transport endpoint is not connected 2019-11-25T16:09:01.741 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:01.736+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:01.741 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:01.737+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:01.830 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:01.826+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:02.030 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:02.025+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:02.382 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:02.378+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:02.398 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:02.393+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:02.496 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:02.490+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:02.650 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:02.644+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:02.745 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:02.740+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:02.748 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:02.744+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:02.826 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:02.822+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:03.045 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:03.038+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:03.380 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:03.376+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:03.426 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:03.417+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:03.456 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:03.451+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:03.677 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:03.673+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:03.732 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:03.727+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:03.732 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:03.728+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:03.817 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:03.813+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:04.091 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:04.082+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:04.363 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:04.359+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:04.395 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:04.391+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:04.440 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:04.437+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:04.641 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:04.636+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:04.724 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:04.720+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:04.727 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:04.724+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:04.818 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:04.814+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:05.064 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:05.060+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:05.369 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:05.365+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:05.372 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:05.369+0000 16949700 -1 osd.4 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:05.374 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:05.370+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:05.417 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:05.411+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:05.660 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:05.656+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:05.721 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:05.717+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:05.722 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:05.719+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:05.809 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:05.803+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:06.018 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:09:06.013+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e282 v282) 2019-11-25T16:09:06.092 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:06.088+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:06.368 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:06.363+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:06.378 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:06.375+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:06.420 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:06.416+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:06.420 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:06.417+0000 16949700 -1 osd.4 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:06.644 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:06.639+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:06.737 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:06.732+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:06.738 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:06.734+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:06.788 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:06.782+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:07.146 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:07.141+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:07.346 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:07.341+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:07.359 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:07.354+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:07.401 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:07.396+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:07.402 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:07.398+0000 16949700 -1 osd.4 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:07.609 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:07.605+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:07.777 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:07.774+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:07.780 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:07.776+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:07.808 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:07.802+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:08.156 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:08.151+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:08.349 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:08.344+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:08.355 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:08.351+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:08.357 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:08.353+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:08.358 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:08.354+0000 16949700 -1 osd.4 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:08.637 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:08.632+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:08.779 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:08.773+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:08.779 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:08.775+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:08.853 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:08.849+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:09.160 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:09.156+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:09.348 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:09.344+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:09.349 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:09.345+0000 16949700 -1 osd.4 282 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:09.399 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:09.396+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:09.401 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:09.397+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:09.665 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:09.661+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:09.811 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:09.801+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:09.812 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:09.802+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:09.877 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:09.873+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:10.214 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:10.209+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:10.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:10.375+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:10.386 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:10.377+0000 16949700 -1 osd.4 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:10.396 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:10.392+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:10.437 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:10.434+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:10.678 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:10.674+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:10.773 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:10.769+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:10.776 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:10.773+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:10.908 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:10.905+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:11.043 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:09:11.035+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e282 v282) 2019-11-25T16:09:11.260 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:11.256+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:11.376 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:11.372+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:11.378 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:11.375+0000 16949700 -1 osd.4 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:11.398 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:11.394+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:11.436 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:11.431+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:11.642 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:11.638+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:11.799 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:11.790+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:11.800 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:11.793+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:11.877 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:11.873+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:12.311 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:12.302+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:12.369 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:12.364+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:12.369 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:12.365+0000 16949700 -1 osd.4 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:12.421 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:12.416+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:12.470 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:12.465+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:12.643 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:12.638+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:12.831 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:12.825+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:12.831 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:12.827+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:12.878 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:12.874+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:13.326 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:13.322+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:13.338 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:13.333+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:13.339 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:13.335+0000 16949700 -1 osd.4 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:13.375 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:13.371+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:13.446 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:13.441+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:13.655 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:13.649+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:13.817 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:13.812+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:13.818 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:13.813+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:13.878 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:13.873+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:14.300 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:14.297+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:14.371 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:14.366+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:14.372 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:14.368+0000 16949700 -1 osd.4 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:14.385 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:14.381+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:14.494 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:14.489+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:14.632 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:14.625+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:14.810 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:14.805+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:14.813 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:14.809+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:14.867 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:14.861+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:15.316 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:15.312+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:15.398 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:15.390+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:15.412 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:15.407+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:15.412 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:15.409+0000 16949700 -1 osd.4 282 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:15.459 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:15.453+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:15.646 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:15.640+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:15.819 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:15.813+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:15.820 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:15.815+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:15.825 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:15.820+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:16.103 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:09:16.049+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e282 v282) 2019-11-25T16:09:16.321 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:16.316+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:16.365 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:16.359+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:16.389 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:16.383+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:16.390 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:16.385+0000 16949700 -1 osd.4 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:16.481 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:16.475+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:16.696 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:16.690+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:16.823 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:16.818+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:16.850 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:16.845+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:16.851 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:16.846+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:17.277 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:17.272+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:17.373 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:17.367+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:17.405 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:17.400+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:17.406 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:17.401+0000 16949700 -1 osd.4 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:17.472 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:17.467+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:17.728 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:17.723+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:17.829 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:17.823+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:17.829 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:17.824+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:17.861 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:17.856+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:18.284 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:18.279+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:18.343 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:18.338+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:18.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:18.380+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:18.386 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:18.381+0000 16949700 -1 osd.4 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:18.498 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:18.493+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:18.683 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:18.678+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:18.803 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:18.797+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:18.804 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:18.798+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:18.890 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:18.885+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:19.268 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:19.263+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:19.313 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:19.308+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:19.432 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:19.426+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:19.433 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:19.428+0000 16949700 -1 osd.4 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:19.507 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:19.504+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:19.679 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:19.673+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:19.841 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:19.837+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:19.842 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:19.838+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:19.909 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:19.905+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:20.294 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:20.289+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:20.302 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:20.298+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:20.408 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:20.404+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:20.411 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:20.407+0000 16949700 -1 osd.4 282 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:20.490 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:20.485+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:20.699 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:20.696+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:20.817 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:20.811+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:20.817 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:20.813+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:20.928 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:20.924+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:21.071 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:09:21.065+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e282 v282) 2019-11-25T16:09:21.292 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:21.287+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:21.304 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:21.296+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:21.375 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:21.370+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:21.376 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:21.371+0000 16949700 -1 osd.4 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:21.459 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:21.454+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:21.663 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:21.659+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:21.793 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:21.788+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:21.793 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:21.789+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:21.979 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:21.975+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:22.263 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:22.257+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:22.275 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:22.271+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:22.394 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:22.390+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:22.400 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:22.396+0000 16949700 -1 osd.4 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:22.470 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:22.466+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:22.629 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:22.624+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:22.765 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:22.761+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:22.767 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:22.763+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:22.963 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:22.959+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:23.276 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:23.271+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:23.327 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:23.322+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:23.363 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:23.357+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:23.364 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:23.359+0000 16949700 -1 osd.4 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:23.438 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:23.434+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:23.603 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:23.598+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:23.800 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:23.795+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:23.800 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:23.796+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:24.015 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:24.011+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:24.238 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:24.232+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:24.375 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:24.369+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:24.404 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:24.399+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:24.407 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:24.402+0000 16949700 -1 osd.4 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:24.449 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:24.445+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:24.566 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:24.561+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:24.834 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:24.830+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:24.837 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:24.834+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:24.986 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:24.981+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:25.297 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:25.287+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:25.398 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:25.389+0000 16949700 -1 osd.4 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507177+0000 front 2019-11-25T16:07:52.566776+0000 (oldest deadline 2019-11-25T16:08:35.860750+0000) 2019-11-25T16:09:25.398 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:25.392+0000 16949700 -1 osd.4 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:25.413 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:09:25.409+0000 16949700 -1 osd.5 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:56.507437+0000 front 2019-11-25T16:07:52.851353+0000 (oldest deadline 2019-11-25T16:08:33.578297+0000) 2019-11-25T16:09:25.476 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:09:25.471+0000 16949700 -1 osd.1 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.563275+0000 front 2019-11-25T16:07:56.507426+0000 (oldest deadline 2019-11-25T16:08:36.867794+0000) 2019-11-25T16:09:25.571 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:25.566+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:25.864 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:25.859+0000 16949700 -1 osd.3 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:08:09.487334+0000 (oldest deadline 2019-11-25T16:08:49.487334+0000) 2019-11-25T16:09:25.865 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:09:25.861+0000 16949700 -1 osd.3 282 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:90936 7.7 7.56ef4467 (undecoded) ondisk+retry+read+known_if_redirected e280) 2019-11-25T16:09:25.965 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:09:25.961+0000 16949700 -1 osd.2 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.279368+0000 front 2019-11-25T16:07:52.562248+0000 (oldest deadline 2019-11-25T16:08:34.642329+0000) 2019-11-25T16:09:26.151 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:09:26.144+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e282 v282) 2019-11-25T16:09:26.287 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:09:26.282+0000 16949700 -1 osd.7 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:52.562735+0000 front 2019-11-25T16:07:52.564063+0000 (oldest deadline 2019-11-25T16:08:35.285275+0000) 2019-11-25T16:09:26.466 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:26.458+0000 16949700 -1 osd.4 283 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:26.581 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:09:26.577+0000 16949700 -1 osd.0 282 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:07:51.965133+0000 front 2019-11-25T16:07:51.903706+0000 (oldest deadline 2019-11-25T16:08:36.365452+0000) 2019-11-25T16:09:27.498 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:27.491+0000 16949700 -1 osd.4 283 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:28.481 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:28.477+0000 16949700 -1 osd.4 284 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:29.496 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:29.491+0000 16949700 -1 osd.4 284 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:30.465 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:30.461+0000 16949700 -1 osd.4 284 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:31.475 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:31.470+0000 16949700 -1 osd.4 284 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:32.504 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:32.499+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:33.558 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:33.553+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:34.513 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:34.508+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:35.548 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:35.543+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:36.526 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:36.517+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:37.539 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:37.533+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:38.520 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:38.515+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:39.560 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:39.555+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:40.554 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:40.550+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:41.544 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:41.538+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:42.515 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:42.511+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:43.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:43.559+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:44.594 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:44.589+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:45.557 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:45.551+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:46.091 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:09:46.070+0000 16949700 -1 osd.6 280 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:113897 6.1 6.c9be1e91 (undecoded) ondisk+read+known_if_redirected e280) 2019-11-25T16:09:46.538 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:46.531+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:47.492 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:47.487+0000 16949700 -1 osd.4 284 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:09:48.776 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:09:48.668+0000 16949700 -1 osd.4 285 get_health_metrics reporting 7 slow ops, oldest is osd_op(client.4500.0:113936 5.3 5:c609908c:::notify.5:head [watch ping cookie 5442140224 gen 1] snapc 0=[] ondisk+write+known_if_redirected e282) 2019-11-25T16:10:05.254 INFO:teuthology.orchestra.run.smithi153.stderr:testSerialization (test.functional.tests.TestFile) ... ok 2019-11-25T16:10:20.469 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:55:25.052 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:10:20.477 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:55:25.060 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:10:20.483 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:55:25.089 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:10:20.487 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:55:25.094 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:10:20.488 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:55:25.094 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:10:20.493 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:55:25.099 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:10:20.739 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:55:25.346 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:10:20.752 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:55:25.335 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:10:20.801 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:55:25.408 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:11:13.301 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:11:13.271+0000 34a6f700 -1 osd.3 287 build_incremental_map_msg missing incremental map 269 2019-11-25T16:11:13.485 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:11:13.467+0000 34a6f700 -1 osd.3 287 build_incremental_map_msg missing incremental map 282 2019-11-25T16:11:14.053 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:14.049+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:14.515 INFO:teuthology.orchestra.run.smithi153.stderr:testStackedOverwrite (test.functional.tests.TestFile) ... ok 2019-11-25T16:11:14.814 INFO:teuthology.orchestra.run.smithi153.stderr:testTooLongName (test.functional.tests.TestFile) ... ok 2019-11-25T16:11:15.056 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:15.051+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:16.103 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:16.098+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:16.347 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:16.341+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:16.613 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:16.607+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:16.758 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:16.746+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:17.146 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:17.141+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:17.333 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:17.326+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:17.359 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:17.355+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:17.601 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:17.596+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:17.770 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:17.765+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:18.139 INFO:teuthology.orchestra.run.smithi153.stderr:testZeroByteFile (test.functional.tests.TestFile) ... ok 2019-11-25T16:11:18.188 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:18.183+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:18.304 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:18.299+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:18.320 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:18.315+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:18.733 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:18.729+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:18.746 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:18.704+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:18.849 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:18.843+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:19.211 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:19.206+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:19.279 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:19.273+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:19.351 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:19.346+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:19.680 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:19.674+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:19.719 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:19.713+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:19.827 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:19.821+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:20.217 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:20.212+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:20.292 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:20.286+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:20.398 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:20.392+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:20.639 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:20.634+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:20.686 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:20.678+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:20.849 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:20.843+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:21.213 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:21.208+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:21.256 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:21.250+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:21.424 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:21.419+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:21.624 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:21.618+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:21.708 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:21.702+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:21.839 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:21.834+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:22.183 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:22.177+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:22.236 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:22.231+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:22.414 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:22.408+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:22.579 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:22.574+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:22.729 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:22.724+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:22.828 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:22.823+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:23.140 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:23.135+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:23.221 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:23.215+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:23.457 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:23.452+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:23.590 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:23.584+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:23.767 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:23.761+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:23.863 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:23.857+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:24.176 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:24.172+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:24.250 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:24.244+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:24.445 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:24.441+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:24.603 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:24.598+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:24.750 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:24.744+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:24.836 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:24.830+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:25.195 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:25.191+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:25.242 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:25.236+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:25.475 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:25.471+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:25.631 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:25.624+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:25.733 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:25.727+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:25.854 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:25.849+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:26.226 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:26.220+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:26.230 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:26.225+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:26.440 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:26.435+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:26.598 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:26.593+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:26.740 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:26.735+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:26.881 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:26.875+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:27.235 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:27.229+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:27.266 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:27.262+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:27.412 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:27.408+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:27.633 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:27.626+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:27.750 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:27.744+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:27.855 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:27.849+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:28.276 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:28.272+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:28.281 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:28.276+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:28.430 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:28.426+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:28.625 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:28.619+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:28.747 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:28.742+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:28.885 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:28.880+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:28.999 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:28.994+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:29.301 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:29.296+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:29.322 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:29.318+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:29.410 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:29.406+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:29.633 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:29.623+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:29.742 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:29.736+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:29.860 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:29.854+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:29.995 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:29.990+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:30.299 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:30.294+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:30.356 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:30.351+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:30.438 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:30.434+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:30.678 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:30.673+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:30.723 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:30.718+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:30.865 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:30.859+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:30.999 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:30.994+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:31.283 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:31.279+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:31.360 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:31.354+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:31.421 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:31.416+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:31.724 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:31.718+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:31.735 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:31.729+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:31.904 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:31.899+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:32.053 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:32.047+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:32.296 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:32.292+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:32.364 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:32.358+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:32.392 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:32.387+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:32.692 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:32.686+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:32.758 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:32.752+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:32.880 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:32.874+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:33.085 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:33.080+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:33.344 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:33.339+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:33.346 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:33.341+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:33.355 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:33.350+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:33.695 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:33.684+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:33.778 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:33.771+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:33.881 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:33.874+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:34.118 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:34.113+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:34.306 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:34.300+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:34.367 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:34.363+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:34.394 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:34.391+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:34.685 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:34.679+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:34.748 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:34.743+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:34.905 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:34.896+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:35.160 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:35.155+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:35.271 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:35.267+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:35.356 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:35.351+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:35.392 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:35.387+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:35.693 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:35.688+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:35.726 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:35.718+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:35.905 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:35.900+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:36.202 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:36.199+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:36.265 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:36.261+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:36.361 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:36.355+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:36.376 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:36.371+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:36.685 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:36.680+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:36.716 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:36.712+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:36.957 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:36.951+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:37.206 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:37.201+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:37.318 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:37.313+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:37.325 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:37.320+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:37.383 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:37.378+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:37.686 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:37.679+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:37.731 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:37.726+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:38.010 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:38.006+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:38.242 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:38.237+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:38.292 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:38.287+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:38.341 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:38.336+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:38.401 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:38.396+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:38.728 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:38.723+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:38.786 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:38.782+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:38.999 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:38.995+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:39.245 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:39.241+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:39.287 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:39.282+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:39.356 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:39.352+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:39.405 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:39.401+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:39.702 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:39.697+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:39.803 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:39.798+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:40.047 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:40.042+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:40.236 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:40.231+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:40.265 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:40.258+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:40.384 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:40.380+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:40.404 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:40.401+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:40.742 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:40.737+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:40.781 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:40.776+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:41.052 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:41.048+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:41.217 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:41.213+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:41.233 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:41.228+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:41.348 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:41.343+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:41.383 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:41.379+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:41.782 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:41.777+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:41.815 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:41.811+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:42.099 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:42.094+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:42.204 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:42.199+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:42.218 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:42.213+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:42.371 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:42.367+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:42.413 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:42.409+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:42.804 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:42.797+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:42.833 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:42.829+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:43.084 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:43.080+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:43.185 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:43.180+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:43.203 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:43.199+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:43.342 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:43.337+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:43.462 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:43.458+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:43.762 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:43.758+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:43.790 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:43.786+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:44.088 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:44.081+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:44.159 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:44.156+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:44.204 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:44.200+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:44.300 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:44.297+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:44.444 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:44.438+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:44.781 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:44.776+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:44.841 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:44.836+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:45.135 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:45.130+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:45.144 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:45.140+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:45.195 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:45.189+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:45.333 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:45.328+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:45.422 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:45.418+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:45.789 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:45.784+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:45.802 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:45.798+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:46.102 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:46.097+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:46.151 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:46.147+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:46.183 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:46.178+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:46.363 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:46.359+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:46.383 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:46.378+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:46.734 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:11:46.729+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 42sec e287 v287) 2019-11-25T16:11:46.819 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:46.815+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:46.859 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:46.853+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:47.077 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:47.074+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:47.188 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:47.183+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:47.196 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:47.192+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:47.331 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:47.326+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:47.403 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:47.398+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:47.844 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:47.838+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:47.858 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:47.853+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:48.076 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:48.070+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:48.182 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:48.177+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:48.238 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:48.232+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:48.293 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:48.288+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:48.371 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:48.367+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:48.808 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:48.803+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:48.885 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:48.878+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:49.059 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:49.052+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:49.177 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:49.172+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:49.268 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:49.262+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:49.330 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:49.324+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:49.412 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:49.408+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:49.775 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:49.769+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:49.855 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:49.849+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:50.095 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:50.089+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:50.160 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:50.154+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:50.301 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:50.296+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:50.318 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:50.314+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:50.459 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:50.455+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:50.773 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:50.768+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:50.862 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:50.856+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:51.056 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:51.051+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:11:51.140 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:11:51.134+0000 16949700 -1 osd.2 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:29.549836+0000 front 2019-11-25T16:10:29.522353+0000 (oldest deadline 2019-11-25T16:11:28.402986+0000) 2019-11-25T16:11:51.282 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:11:51.276+0000 16949700 -1 osd.7 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.854148+0000 front 2019-11-25T16:10:33.291305+0000 (oldest deadline 2019-11-25T16:11:16.260982+0000) 2019-11-25T16:11:51.339 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:11:51.332+0000 16949700 -1 osd.0 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.754605+0000 front 2019-11-25T16:10:33.049914+0000 (oldest deadline 2019-11-25T16:11:13.707414+0000) 2019-11-25T16:11:51.513 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:11:51.506+0000 16949700 -1 osd.1 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:31.781271+0000 front 2019-11-25T16:10:31.738298+0000 (oldest deadline 2019-11-25T16:11:16.624234+0000) 2019-11-25T16:11:51.770 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:11:51.761+0000 16949700 -1 osd.4 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:32.066044+0000 front 2019-11-25T16:10:32.166779+0000 (oldest deadline 2019-11-25T16:11:15.751858+0000) 2019-11-25T16:11:51.843 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:11:51.831+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.3 [v2:172.21.15.153:6826/9696,v1:172.21.15.153:6827/9696] for 42sec e287 v287) 2019-11-25T16:11:51.885 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:11:51.879+0000 16949700 -1 osd.5 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:30.555893+0000 front 2019-11-25T16:10:30.511030+0000 (oldest deadline 2019-11-25T16:11:15.832175+0000) 2019-11-25T16:11:52.100 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:11:52.095+0000 16949700 -1 osd.6 287 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:10:28.412593+0000 front 2019-11-25T16:10:28.554303+0000 (oldest deadline 2019-11-25T16:11:18.795243+0000) 2019-11-25T16:12:38.956 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:12:38.953+0000 114a8700 -1 osd.2 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.125 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:12:39.067+0000 34a6f700 -1 osd.2 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.162 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:12:39.093+0000 3aa7b700 -1 osd.2 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.176 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:12:38.985+0000 36a73700 -1 osd.1 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.186 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:12:39.040+0000 35a71700 -1 osd.1 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.218 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:12:39.192+0000 11ca9700 -1 osd.2 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.243 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:12:39.230+0000 3aa7b700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.257 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:12:39.243+0000 37274700 -1 osd.2 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.278 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:12:39.230+0000 38a77700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.811 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:12:39.607+0000 38276700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:39.959 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:12:39.857+0000 3a27a700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:40.124 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:12:40.079+0000 38276700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:40.336 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:12:40.331+0000 11ca9700 -1 osd.6 291 build_incremental_map_msg missing incremental map 291 2019-11-25T16:12:41.425 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:12:41.419+0000 35a71700 -1 osd.3 292 build_incremental_map_msg missing incremental map 292 2019-11-25T16:12:41.482 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:12:41.475+0000 35270700 -1 osd.3 292 build_incremental_map_msg missing incremental map 292 2019-11-25T16:12:41.539 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:12:41.506+0000 36272700 -1 osd.3 292 build_incremental_map_msg missing incremental map 292 2019-11-25T16:12:42.364 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:12:42.039+0000 34a6f700 -1 osd.3 293 build_incremental_map_msg missing incremental map 292 2019-11-25T16:12:50.286 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:12:50.264+0000 36a73700 -1 osd.3 293 build_incremental_map_msg missing incremental map 292 2019-11-25T16:12:51.961 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:12:51.933+0000 37274700 -1 osd.3 293 build_incremental_map_msg missing incremental map 292 2019-11-25T16:13:09.687 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:09.684+0000 16949700 -1 osd.3 293 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:10.667 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:10.662+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:11.717 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:11.712+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:12.687 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:12.683+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:13.663 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:13.658+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:14.731 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:14.720+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:15.720 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:15.716+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:16.729 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:16.725+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:17.738 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:17.734+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:18.775 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:18.770+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:19.814 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:19.810+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:20.840 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:20.835+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:21.840 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:21.837+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:22.816 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:22.812+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:23.856 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:23.849+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:24.881 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:24.877+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:25.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:25.915+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:26.909 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:26.905+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:27.891 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:27.887+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:28.890 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:28.886+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:29.882 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:29.878+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:30.925 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:30.920+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:31.894 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:31.889+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:32.934 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:32.930+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:33.944 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:33.940+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:34.938 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:34.934+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:35.011 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:35.006+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:35.735 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:35.728+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:35.898 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:35.893+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:35.898 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:35.894+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:35.946 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:35.941+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:36.036 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:36.033+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:36.719 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:36.713+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:36.752 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:36.747+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:36.939 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:36.932+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:36.940 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:36.935+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:36.960 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:36.955+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:37.050 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:37.044+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:37.697 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:37.691+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:37.788 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:37.782+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:37.910 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:37.905+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:37.949 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:37.943+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:37.964 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:37.957+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:37.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:37.960+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:38.074 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:38.068+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:38.676 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:38.671+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:38.677 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:38.672+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:38.807 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:38.797+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:38.933 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:38.926+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:38.972 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:38.967+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:38.973 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:38.968+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:38.992 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:38.985+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:39.087 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:39.082+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:39.672 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:39.668+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:39.673 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:39.669+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:39.814 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:39.809+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:39.910 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:39.903+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:39.948 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:39.942+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:39.949 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:39.944+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:39.982 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:39.977+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:40.060 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:40.054+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:40.659 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:40.653+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:40.692 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:40.687+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:40.832 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:40.827+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:40.922 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:40.916+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:40.923 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:40.918+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:40.946 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:40.941+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:40.957 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:40.952+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:41.101 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:41.095+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:41.643 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:41.638+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:41.657 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:41.651+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:41.800 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:41.794+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:41.903 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:41.898+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:41.913 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:41.907+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:41.916 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:41.911+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:42.011 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:42.006+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:42.155 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:42.149+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:42.693 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:42.687+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:42.694 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:42.688+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:42.773 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:42.769+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:42.914 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:42.909+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:42.915 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:42.911+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:42.930 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:42.924+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:43.014 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:43.010+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:43.178 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:43.174+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:43.652 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:43.646+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:43.683 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:43.678+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:43.737 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:43.733+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:43.927 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:43.921+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:43.962 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:43.956+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:43.962 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:43.958+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:44.018 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:44.013+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:44.207 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:44.200+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:44.624 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:44.618+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:44.703 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:44.697+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:44.788 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:44.783+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:44.899 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:44.894+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:44.967 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:44.963+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:44.969 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:44.966+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:45.022 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:45.018+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:45.246 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:45.241+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:45.668 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:45.662+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:45.715 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:45.709+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:45.788 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:45.782+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:45.917 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:45.911+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:45.982 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:45.975+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:45.983 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:45.978+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:46.056 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:46.053+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:46.256 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:46.250+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:46.644 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:46.640+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:46.758 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:46.752+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:46.759 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:46.755+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:46.900 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:46.895+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:46.960 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:46.955+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:46.960 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:46.957+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:47.027 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:47.022+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:47.273 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:47.268+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:47.659 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:47.653+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:47.785 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:47.780+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:47.809 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:47.805+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:47.874 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:47.869+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:47.987 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:47.982+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:47.991 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:47.986+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:47.991 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:47.987+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:48.316 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:48.311+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:48.692 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:48.687+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:48.811 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:48.807+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:48.850 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:48.846+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:48.871 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:48.866+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:49.034 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:49.030+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:49.034 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:49.031+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:49.036 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:49.033+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:49.338 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:49.333+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:49.684 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:49.679+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:49.795 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:49.790+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:49.832 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:49.828+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:49.867 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:49.862+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:50.036 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:50.031+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:50.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:50.075+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:50.082 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:50.077+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:50.386 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:50.381+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:50.689 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:13:50.684+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.029022+0000 front 2019-11-25T16:12:56.030702+0000 (oldest deadline 2019-11-25T16:13:36.479167+0000) 2019-11-25T16:13:50.840 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:50.834+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:50.849 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:50.845+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:50.913 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:50.908+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:51.051 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:51.046+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:51.111 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:51.107+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:51.116 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:51.111+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:51.346 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:51.342+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:51.823 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:51.819+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:51.867 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:51.862+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:51.892 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:51.886+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:52.034 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:52.030+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:52.118 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:52.114+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:52.120 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:52.117+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:52.341 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:52.336+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:52.800 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:52.796+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:52.858 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:52.853+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:52.897 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:52.891+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:53.015 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:53.011+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:53.085 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:53.081+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:53.089 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:53.085+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:53.335 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:53.331+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:53.769 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:53.764+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:53.897 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:53.892+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:53.936 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:53.930+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:53.988 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:53.984+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:54.057 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:54.051+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:54.058 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:54.055+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:54.380 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:54.373+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:54.771 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:54.767+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:54.945 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:54.940+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:54.976 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:54.971+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:55.022 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:55.017+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:55.077 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:55.073+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:55.080 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:55.077+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:55.366 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:55.362+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:55.806 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:55.802+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:55.946 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:55.940+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:55.988 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:55.984+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:56.013 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:56.009+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:56.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:56.075+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:56.082 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:56.077+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:56.351 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:56.346+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:56.810 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:56.806+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:56.907 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:56.901+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:57.011 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:57.006+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:57.041 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:57.036+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:57.127 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:57.123+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:57.128 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:57.124+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:57.319 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:57.314+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:57.780 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:57.776+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:57.922 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:57.917+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:58.051 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:58.047+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:58.093 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:58.087+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:58.136 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:58.132+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:58.136 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:58.133+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:58.351 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:13:58.346+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:51.638663+0000 front 2019-11-25T16:12:51.540633+0000 (oldest deadline 2019-11-25T16:13:34.767023+0000) 2019-11-25T16:13:58.770 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:58.765+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:58.924 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:58.919+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:59.017 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:59.013+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:13:59.142 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:13:59.137+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:13:59.155 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:59.149+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:13:59.155 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:13:59.151+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:13:59.766 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:13:59.760+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:13:59.966 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:13:59.961+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:13:59.990 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:13:59.986+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:00.164 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:00.160+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:00.168 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:00.164+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:00.173 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:00.165+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:00.755 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:00.751+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:00.951 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:00.946+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:00.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:00.974+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:01.122 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:01.117+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:01.123 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:01.119+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:01.191 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:01.186+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:01.807 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:01.803+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:01.969 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:01.964+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:01.976 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:01.970+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:02.110 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:02.106+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:02.114 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:02.110+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:02.181 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:02.175+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:02.808 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:02.804+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:02.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:02.972+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:03.027 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:03.023+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:03.083 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:03.079+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:03.088 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:03.082+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:03.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:03.225+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:03.789 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:03.785+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:04.002 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:03.997+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:04.022 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:04.017+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:04.127 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:04.121+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:04.129 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:04.123+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:04.199 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:04.194+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:04.838 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:04.833+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:05.035 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:05.029+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:05.067 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:05.062+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:05.176 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:05.170+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:05.177 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:05.172+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:05.223 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:05.218+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:05.793 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:05.788+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:06.015 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:06.011+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:06.055 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:06.050+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:06.155 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:06.151+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:06.158 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:06.154+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:06.215 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:06.209+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:06.849 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:06.840+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:07.009 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:07.005+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:07.103 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:07.099+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:07.147 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:07.142+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:07.147 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:07.143+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:07.250 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:07.246+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:07.867 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:07.863+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:07.965 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:07.961+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:08.095 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:08.090+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:08.140 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:08.136+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:08.141 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:08.137+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:08.267 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:08.263+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:08.915 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:14:08.911+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.163180+0000 front 2019-11-25T16:12:56.025910+0000 (oldest deadline 2019-11-25T16:13:35.329197+0000) 2019-11-25T16:14:09.007 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:09.004+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:12:51.677018+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:09.116 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:09.112+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:12:52.860902+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:09.118 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:09.115+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:09.135 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:09.131+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:09.293 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:14:09.290+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.032326+0000 front 2019-11-25T16:12:52.463328+0000 (oldest deadline 2019-11-25T16:13:37.651011+0000) 2019-11-25T16:14:09.578 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:14:09.563+0000 3b27c700 -1 osd.6 293 build_incremental_map_msg missing incremental map 291 2019-11-25T16:14:09.588 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:14:09.576+0000 3b27c700 -1 osd.6 293 build_incremental_map_msg missing incremental map 291 2019-11-25T16:14:10.000 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:09.996+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:14:09.601698+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:10.151 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:10.147+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:10.248 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:10.229+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:14:09.631432+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:10.256 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:10.250+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:10.985 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:10.980+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:14:09.601698+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:11.120 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:14:11.116+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:56.030180+0000 front 2019-11-25T16:12:56.046354+0000 (oldest deadline 2019-11-25T16:13:37.054187+0000) 2019-11-25T16:14:11.388 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:11.301+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:14:09.631432+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:11.412 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:11.396+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:11.956 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:14:11.952+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:53.161379+0000 front 2019-11-25T16:14:09.601698+0000 (oldest deadline 2019-11-25T16:13:35.922132+0000) 2019-11-25T16:14:12.505 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:14:12.478+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 46sec e293 v293) 2019-11-25T16:14:12.583 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:12.503+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:12:52.244560+0000 front 2019-11-25T16:14:09.631432+0000 (oldest deadline 2019-11-25T16:13:35.450957+0000) 2019-11-25T16:14:12.800 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:12.547+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:13.953 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:13.819+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:16.046 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:14:15.624+0000 16949700 -1 osd.3 293 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7.3e1ea487 (undecoded) ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:14:20.612 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:59:25.195 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:20.617 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:59:25.223 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:20.618 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:59:25.201 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:20.622 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:59:25.228 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:20.634 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:59:25.241 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:20.640 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:59:25.247 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:20.888 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:59:25.494 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:20.896 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:59:25.479 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:20.970 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:59:25.576 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.776 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:59:27.359 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.780 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:59:27.363 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.805 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:06:59:27.388 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.805 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:59:27.411 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.809 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:59:27.415 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.824 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:59:27.430 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.827 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:06:59:27.433 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.838 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:59:27.444 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:14:22.856 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:06:59:27.462 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:15:51.155 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:51.151+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:51.597 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:51.591+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:15:52.125 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:52.121+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:52.347 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:15:52.343+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:15:52.391 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:15:52.385+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:15:52.424 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:15:52.420+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:15:52.584 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:52.580+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:15:53.143 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:53.137+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:53.379 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:15:53.374+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:15:53.393 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:15:53.389+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:15:53.436 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:15:53.430+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:15:53.586 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:53.581+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:15:54.143 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:54.138+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:54.393 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:15:54.388+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:15:54.397 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:15:54.392+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:15:54.420 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:15:54.414+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:15:54.611 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:54.607+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:15:55.189 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:55.184+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:55.224 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:15:55.218+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:15:55.386 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:15:55.381+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:15:55.404 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:15:55.399+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:15:55.410 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:15:55.404+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:15:55.597 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:55.591+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:15:56.182 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:15:56.177+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:15:56.235 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:56.230+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:56.394 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:15:56.390+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:15:56.428 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:15:56.423+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:15:56.433 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:15:56.428+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:15:56.574 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:56.569+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:15:57.158 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:15:57.153+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:15:57.217 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:57.211+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:57.399 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:15:57.394+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:15:57.425 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:15:57.420+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:15:57.443 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:15:57.438+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:15:57.602 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:57.597+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:15:58.191 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:58.187+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:58.206 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:15:58.200+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:15:58.413 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:15:58.408+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:15:58.428 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:15:58.423+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:15:58.431 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:15:58.426+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:15:58.620 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:58.615+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:15:59.187 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:15:59.183+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:15:59.236 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:15:59.231+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:15:59.399 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:15:59.394+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:15:59.411 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:15:59.407+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:15:59.472 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:15:59.468+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:15:59.582 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:15:59.578+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:00.182 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:00.177+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:00.230 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:00.226+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:00.357 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:00.352+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:00.448 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:00.444+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:00.470 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:00.466+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:00.546 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:00.542+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:01.178 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:01.172+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:01.211 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:01.206+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:01.377 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:01.370+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:01.426 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:01.420+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:01.438 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:01.433+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:01.517 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:01.512+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:02.152 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:02.146+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:02.246 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:02.242+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:02.396 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:02.390+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:02.419 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:02.414+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:02.426 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:02.421+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:02.515 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:02.509+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:03.140 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:03.135+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:03.285 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:03.280+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:03.408 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:03.403+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:03.416 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:03.411+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:03.423 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:03.418+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:03.472 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:03.466+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:04.136 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:04.129+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:04.306 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:04.300+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:04.382 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:04.377+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:04.383 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:04.379+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:04.411 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:04.404+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:04.457 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:04.452+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:05.154 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:05.148+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:05.269 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:05.264+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:05.402 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:05.396+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:05.410 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:05.405+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:05.419 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:05.414+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:05.474 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:05.469+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:06.132 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:06.127+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:06.235 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:06.231+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:06.433 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:06.428+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:06.435 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:06.431+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:06.443 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:06.437+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:06.497 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:06.492+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:07.154 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:07.148+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:07.246 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:07.241+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:07.399 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:07.392+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:07.440 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:07.435+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:07.484 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:07.478+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:07.530 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:07.524+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:07.629 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:07.624+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:08.126 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:08.120+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:08.252 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:08.248+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:08.451 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:08.445+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:08.490 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:08.486+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:08.527 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:08.521+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:08.530 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:08.525+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:08.595 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:08.590+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:09.115 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:09.110+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:09.221 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:09.217+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:09.410 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:09.404+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:09.476 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:09.473+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:09.492 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:09.486+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:09.511 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:09.504+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:09.633 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:09.628+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:10.116 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:10.110+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:10.267 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:10.264+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:10.388 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:10.382+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:10.506 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:10.498+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:10.519 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:10.515+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:10.546 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:10.541+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:10.617 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:10.611+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:11.110 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:16:11.105+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:11.160296+0000 front 2019-11-25T16:15:09.284085+0000 (oldest deadline 2019-11-25T16:15:50.918201+0000) 2019-11-25T16:16:11.248 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:11.243+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:11.431 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:11.426+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:11.521 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:11.515+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:11.543 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:11.538+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:11.575 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:11.569+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:11.581 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:11.576+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:12.277 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:12.272+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:12.424 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:12.419+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:12.508 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:12.503+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:12.548 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:12.543+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:12.559 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:12.553+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:12.616 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:12.610+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:13.281 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:13.276+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:13.420 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:13.414+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:13.484 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:13.480+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:13.522 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:13.516+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:13.584 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:13.576+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:13.637 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:13.631+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:14.322 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:14.317+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:14.387 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:14.381+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:14.453 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:16:14.448+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.527875+0000 front 2019-11-25T16:15:06.512442+0000 (oldest deadline 2019-11-25T16:15:51.783926+0000) 2019-11-25T16:16:14.514 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:14.507+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:14.631 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:14.625+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:14.644 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:14.638+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:14.784 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:14.780+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:15.285 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:15.279+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:15.404 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:16:15.397+0000 16949700 -1 osd.4 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:09.030029+0000 front 2019-11-25T16:15:08.814051+0000 (oldest deadline 2019-11-25T16:15:52.274476+0000) 2019-11-25T16:16:15.534 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:15.528+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:15.608 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:16:15.602+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.949962+0000 front 2019-11-25T16:15:10.935897+0000 (oldest deadline 2019-11-25T16:15:51.557383+0000) 2019-11-25T16:16:15.621 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:16:15.610+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:08.847688+0000 front 2019-11-25T16:15:08.940701+0000 (oldest deadline 2019-11-25T16:15:51.525110+0000) 2019-11-25T16:16:15.797 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:15.791+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:16.269 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:16:16.262+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:10.491540+0000 front 2019-11-25T16:15:10.511419+0000 (oldest deadline 2019-11-25T16:15:55.008565+0000) 2019-11-25T16:16:16.541 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:16.535+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:16.760 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:16.755+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:17.555 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:16:17.548+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.153:6830 osd.3 since back 2019-11-25T16:15:06.973357+0000 front 2019-11-25T16:15:06.760132+0000 (oldest deadline 2019-11-25T16:16:07.458460+0000) 2019-11-25T16:16:17.718 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:17.714+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:18.679 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:18.674+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:19.716 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:19.711+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:20.707 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:20.703+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:21.733 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:21.728+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:22.747 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:22.742+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:23.757 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:23.752+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:24.734 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:24.730+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:25.802 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:25.796+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:26.795 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:26.791+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:27.798 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:27.794+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:28.814 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:28.811+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:16:29.776 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:16:29.772+0000 16949700 -1 osd.3 293 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4500.0:114538 7.7 7:e125787c:::pubsub.user.foo.client.0.bucket.1f3bfae86631470a9cb1682550589307%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.655:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e291) 2019-11-25T16:17:27.989 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:27.984+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:28.246 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:28.242+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:29.031 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:29.025+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:29.247 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:29.243+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:29.850 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:29.845+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:29.987 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:29.983+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:30.084 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:30.078+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:30.115 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:30.109+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:30.241 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:30.236+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:30.402 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:30.398+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:17:30.889 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:30.885+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:30.996 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:30.987+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:31.059 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:31.055+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:31.090 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:31.087+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:31.241 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:31.236+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:31.424 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:31.419+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:17:31.642 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:17:31.639+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.740950+0000 front 2019-11-25T16:16:47.745477+0000 (oldest deadline 2019-11-25T16:17:31.311063+0000) 2019-11-25T16:17:31.875 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:31.869+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:32.009 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:32.004+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:32.059 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:32.054+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:32.096 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:32.092+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:32.215 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:32.210+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:32.404 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:32.399+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:17:32.620 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:17:32.615+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.740950+0000 front 2019-11-25T16:16:47.745477+0000 (oldest deadline 2019-11-25T16:17:31.311063+0000) 2019-11-25T16:17:32.916 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:32.912+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:33.022 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:33.015+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:33.074 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:33.070+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:33.078 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:33.073+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:33.248 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:33.243+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:33.414 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:33.410+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:17:33.606 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:17:33.601+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.740950+0000 front 2019-11-25T16:16:47.745477+0000 (oldest deadline 2019-11-25T16:17:31.311063+0000) 2019-11-25T16:17:33.883 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:33.878+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:34.002 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:33.997+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:34.069 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:34.065+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:34.111 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:34.106+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:34.269 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:34.263+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:34.426 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:34.422+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:17:34.630 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:17:34.626+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.740950+0000 front 2019-11-25T16:16:47.745477+0000 (oldest deadline 2019-11-25T16:17:31.311063+0000) 2019-11-25T16:17:34.908 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:34.904+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:34.999 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:34.995+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:35.074 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:35.071+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:35.138 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:35.130+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:35.311 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:35.307+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:35.401 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:35.397+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:17:35.620 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:17:35.616+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.740950+0000 front 2019-11-25T16:16:47.745477+0000 (oldest deadline 2019-11-25T16:17:31.311063+0000) 2019-11-25T16:17:35.944 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:35.940+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:36.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:36.030+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:36.106 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:36.102+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:36.113 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:36.110+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:36.353 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:36.349+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:36.387 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:36.381+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:17:36.672 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:17:36.668+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.740950+0000 front 2019-11-25T16:16:47.745477+0000 (oldest deadline 2019-11-25T16:17:31.311063+0000) 2019-11-25T16:17:36.967 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:36.962+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:37.025 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:37.020+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:37.112 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:37.107+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:37.159 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:37.154+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:37.344 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:37.339+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:17:37.345 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:37.341+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:37.687 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:17:37.683+0000 16949700 -1 osd.6 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.740950+0000 front 2019-11-25T16:16:47.745477+0000 (oldest deadline 2019-11-25T16:17:31.311063+0000) 2019-11-25T16:17:37.968 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:17:37.963+0000 16949700 -1 osd.1 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:48.314499+0000 front 2019-11-25T16:16:47.727755+0000 (oldest deadline 2019-11-25T16:17:29.044003+0000) 2019-11-25T16:17:38.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:17:38.061+0000 16949700 -1 osd.0 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.027760+0000 front 2019-11-25T16:16:47.970713+0000 (oldest deadline 2019-11-25T16:17:27.911672+0000) 2019-11-25T16:17:38.092 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:17:38.088+0000 16949700 -1 osd.3 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:44.939748+0000 front 2019-11-25T16:16:44.880102+0000 (oldest deadline 2019-11-25T16:17:29.492995+0000) 2019-11-25T16:17:38.175 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:17:38.170+0000 16949700 -1 osd.5 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:45.290199+0000 front 2019-11-25T16:16:46.744791+0000 (oldest deadline 2019-11-25T16:17:29.274359+0000) 2019-11-25T16:17:38.334 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:17:38.327+0000 16949700 -1 osd.2 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:47.853755+0000 front 2019-11-25T16:16:42.471408+0000 (oldest deadline 2019-11-25T16:17:27.752202+0000) 2019-11-25T16:17:38.395 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:17:38.391+0000 16949700 -1 osd.7 293 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:16:46.801731+0000 front 2019-11-25T16:16:46.796530+0000 (oldest deadline 2019-11-25T16:17:29.351465+0000) 2019-11-25T16:18:38.849 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:03:43.432 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:38.851 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:03:43.434 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:38.925 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:03:43.531 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:38.932 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:03:43.538 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:39.009 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:03:43.615 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:39.016 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:03:43.622 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:39.093 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:03:43.675 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:39.174 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:03:43.780 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:39.263 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:03:43.869 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:18:41.269 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:18:41.263+0000 38a77700 -1 osd.3 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.387 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:18:41.364+0000 39278700 -1 osd.3 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:18:41.393+0000 35a71700 -1 osd.3 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.441 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:18:41.410+0000 38a77700 -1 osd.2 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.500 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:18:41.279+0000 37a75700 -1 osd.5 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.516 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:18:41.313+0000 3a27a700 -1 osd.5 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.550 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:18:41.518+0000 36272700 -1 osd.3 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.564 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:18:41.393+0000 3426e700 -1 osd.5 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.619 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:18:41.502+0000 39278700 -1 osd.2 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.702 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:18:41.635+0000 3a27a700 -1 osd.6 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.755 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:18:41.699+0000 38276700 -1 osd.6 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:41.860 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:18:41.855+0000 37274700 -1 osd.5 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:42.154 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:18:42.046+0000 3a27a700 -1 osd.6 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:18:42.315 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:18:42.187+0000 37274700 -1 osd.6 297 build_incremental_map_msg missing incremental map 297 2019-11-25T16:19:12.888 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:12.885+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:13.917 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:13.913+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:14.935 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:14.932+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:15.929 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:15.926+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:16.946 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:16.943+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:17.923 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:17.920+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:18.936 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:18.933+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:19.974 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:19.971+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:21.022 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:21.018+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:22.055 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:22.051+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:22.062 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:22.058+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:23.010 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:23.005+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:23.014 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:23.009+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:23.110 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:23.106+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:23.201 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:23.154+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:23.923 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:23.918+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:24.019 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:24.014+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:24.019 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:24.015+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:24.070 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:24.064+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:24.209 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:24.203+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:24.530 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:24.526+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:24.846 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:24.842+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:24.938 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:24.933+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:24.995 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:24.989+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:24.996 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:24.991+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:25.113 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:25.109+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:25.185 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:25.180+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:25.297 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:25.291+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:25.546 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:25.534+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:25.895 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:25.889+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:25.932 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:25.927+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:25.990 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:25.984+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:25.991 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:25.986+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:26.076 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:26.072+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:26.168 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:26.163+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:26.305 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:26.301+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:26.583 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:26.579+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:26.932 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:26.927+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:26.947 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:26.941+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:26.948 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:26.943+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:26.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:26.973+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:27.099 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:27.094+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:27.175 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:27.169+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:27.472 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:27.462+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:27.615 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:27.611+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:27.917 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:27.912+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:27.966 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:27.962+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:27.969 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:27.964+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:27.970 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:27.965+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:28.121 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:28.115+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:28.215 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:28.211+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:28.432 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:28.428+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:28.606 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:28.601+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:28.967 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:28.963+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:28.992 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:28.977+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:28.999 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:28.995+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:29.001 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:28.997+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:29.153 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:29.148+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:29.231 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:29.226+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:29.481 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:29.474+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:29.610 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:29.606+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:29.982 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:29.977+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:29.991 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:29.986+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:29.993 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:29.989+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:29.998 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:29.994+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:30.167 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:30.162+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:30.247 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:30.243+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:30.486 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:30.481+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:30.663 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:30.657+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:30.997 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:30.992+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:30.999 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:30.994+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:31.005 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:31.000+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:31.006 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:31.001+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:31.190 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:31.186+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:31.209 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:31.206+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:31.532 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:31.527+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:31.649 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:31.644+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:32.000 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:31.995+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:32.002 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:31.998+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:32.042 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:32.038+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:32.050 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:32.045+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:32.163 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:32.159+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:32.211 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:32.207+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:32.495 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:32.489+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:32.699 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:32.695+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:33.015 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:33.010+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:33.027 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:33.022+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:33.027 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:33.023+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:33.037 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:33.033+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:33.201 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:33.197+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:33.255 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:33.250+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:33.544 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:33.539+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:33.683 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:33.679+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:34.020 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:34.015+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:34.039 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:34.034+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:34.039 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:34.035+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:34.058 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:34.054+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:34.202 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:34.198+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:34.225 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:34.217+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:34.573 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:34.568+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:34.642 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:34.638+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:34.979 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:34.975+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:35.046 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:35.040+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:35.046 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:35.042+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:35.060 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:35.056+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:35.177 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:35.173+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:35.187 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:35.182+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:35.539 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:35.534+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:35.634 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:35.629+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:35.938 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:35.933+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:36.036 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:36.032+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:36.039 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:36.033+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:36.077 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:36.073+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:36.169 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:36.164+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:36.198 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:36.193+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:36.515 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:36.510+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:36.656 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:36.651+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:36.967 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:36.961+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:37.060 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:37.056+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:37.062 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:37.058+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:37.107 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:37.103+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:37.155 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:37.152+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:37.240 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:37.236+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:37.484 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:37.478+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:37.703 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:37.698+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:37.995 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:37.989+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:38.093 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:38.088+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:38.094 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:38.091+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:38.127 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:38.123+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:38.200 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:38.195+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:38.237 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:38.232+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:38.488 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:38.483+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:38.676 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:38.670+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:38.953 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:38.948+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:39.091 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:39.086+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:39.092 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:39.087+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:39.107 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:39.103+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:39.238 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:39.233+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:39.240 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:39.232+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:39.503 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:39.498+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:39.670 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:39.666+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:39.916 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:39.911+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:40.067 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:40.063+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:40.071 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:40.065+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:40.071 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:40.066+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:40.239 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:40.235+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:40.253 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:40.248+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:40.544 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:40.539+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:40.666 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:40.661+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:40.880 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:40.875+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:41.099 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:41.094+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:41.101 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:41.097+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:41.119 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:41.115+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:41.259 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:41.255+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:41.270 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:41.266+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:41.503 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:41.497+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:41.662 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:41.657+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:41.868 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:41.863+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:42.096 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:42.090+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:42.097 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:42.092+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:42.147 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:42.141+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:42.291 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:42.286+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:42.310 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:42.306+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:42.555 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:42.550+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:42.783 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:42.756+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:42.889 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:42.885+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:43.111 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:43.106+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:43.113 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:43.109+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:43.200 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:43.195+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:43.276 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:43.272+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:43.311 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:43.307+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:43.532 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:43.524+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:43.766 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:43.762+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:43.941 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:43.936+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:44.091 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:44.086+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:44.094 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:44.089+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:44.156 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:04:48.740 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.164 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:04:48.747 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.186 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:44.181+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:44.236 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:04:48.842 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.238 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:04:48.844 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.247 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:04:48.853 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.248 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:04:48.854 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.297 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:04:48.880 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.317 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:44.313+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:44.323 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:44.314+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:44.329 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:04:48.935 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.336 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:04:48.942 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:19:44.519 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:44.502+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:44.777 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:44.771+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:44.980 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:44.975+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:45.097 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:45.091+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:45.097 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:45.092+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:45.242 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:45.237+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:45.343 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:45.338+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:45.371 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:45.366+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:45.518 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:45.513+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:45.743 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:45.738+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:45.989 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:45.984+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:46.061 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:46.055+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:46.062 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:46.057+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:46.236 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:46.231+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:46.300 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:46.294+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:46.352 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:46.347+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:46.604 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:46.598+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:46.750 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:46.745+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:47.041 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:47.036+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:47.095 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:47.089+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:47.096 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:47.091+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:47.269 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:47.263+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:47.334 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:47.329+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:47.365 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:47.360+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:47.640 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:47.634+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:47.762 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:47.756+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:48.056 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:48.051+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:48.057 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:48.052+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:48.074 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:48.070+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:48.286 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:48.280+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:48.347 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:48.341+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:48.356 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:48.350+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:48.673 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:48.667+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:48.778 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:48.773+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:49.019 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:49.013+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:49.020 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:49.015+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:49.043 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:49.037+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:49.264 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:49.257+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:49.310 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:49.305+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:49.347 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:49.342+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:49.672 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:49.667+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:49.775 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:49.770+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:49.977 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:49.972+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:49.978 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:49.974+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:50.003 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:49.998+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:50.220 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:50.215+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:50.331 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:50.326+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:50.390 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:50.384+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:50.658 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:50.653+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:50.767 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:50.762+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:51.010 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:51.006+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:51.013 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:51.009+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:51.017 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:51.012+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:51.226 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:51.220+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:51.356 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:51.350+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:51.360 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:51.352+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:51.777 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:51.772+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:51.784 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:51.769+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:52.026 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:52.022+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:52.062 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:52.056+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:52.062 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:52.057+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:52.255 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:52.251+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:52.354 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:52.350+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:52.379 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:52.376+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:52.761 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:52.757+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:52.802 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:52.797+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:52.995 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:52.990+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:53.041 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:53.035+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:53.041 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:53.036+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:53.273 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:19:53.268+0000 16949700 -1 osd.3 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.391801+0000 front 2019-11-25T16:18:39.849908+0000 (oldest deadline 2019-11-25T16:19:23.835902+0000) 2019-11-25T16:19:53.366 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:19:53.361+0000 16949700 -1 osd.0 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.953129+0000 front 2019-11-25T16:18:38.951517+0000 (oldest deadline 2019-11-25T16:19:21.679213+0000) 2019-11-25T16:19:53.401 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:53.396+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:53.765 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:19:53.760+0000 16949700 -1 osd.5 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.847029+0000 front 2019-11-25T16:18:37.480127+0000 (oldest deadline 2019-11-25T16:19:21.964845+0000) 2019-11-25T16:19:53.769 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:19:53.763+0000 16949700 -1 osd.1 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:39.625428+0000 front 2019-11-25T16:18:39.923802+0000 (oldest deadline 2019-11-25T16:19:24.284869+0000) 2019-11-25T16:19:54.052 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:54.048+0000 16949700 -1 osd.4 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.270386+0000 front 2019-11-25T16:18:40.279014+0000 (oldest deadline 2019-11-25T16:19:22.107138+0000) 2019-11-25T16:19:54.055 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:19:54.050+0000 16949700 -1 osd.4 298 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e297) 2019-11-25T16:19:54.071 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:19:54.063+0000 16949700 -1 osd.7 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:38.845844+0000 front 2019-11-25T16:18:39.738765+0000 (oldest deadline 2019-11-25T16:19:23.002190+0000) 2019-11-25T16:19:54.153 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:19:54.147+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e298 v298) 2019-11-25T16:19:54.404 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:19:54.399+0000 16949700 -1 osd.2 298 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:18:40.271588+0000 front 2019-11-25T16:18:40.272444+0000 (oldest deadline 2019-11-25T16:19:24.332974+0000) 2019-11-25T16:19:57.211 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:19:57.206+0000 34a6f700 -1 osd.6 298 build_incremental_map_msg missing incremental map 297 2019-11-25T16:19:57.253 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:19:57.234+0000 38a77700 -1 osd.6 298 build_incremental_map_msg missing incremental map 297 2019-11-25T16:21:07.053 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:07.049+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:08.097 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:08.092+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:08.125 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:08.120+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:09.074 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:09.069+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:09.153 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:09.147+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:09.660 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:09.653+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:09.705 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:09.700+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:10.043 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:10.037+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:10.177 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:10.170+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:10.182 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:10.176+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:10.715 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:10.709+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:10.742 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:10.736+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:11.047 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:11.043+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:11.141 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:11.136+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:11.162 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:11.156+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:11.209 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:11.204+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:11.711 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:11.706+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:11.748 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:11.743+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:11.925 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:11.921+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:12.005 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:12.000+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:12.116 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:12.111+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:12.142 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:12.136+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:12.256 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:12.251+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:12.730 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:12.725+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:12.784 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:12.779+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:12.941 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:12.935+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:12.993 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:12.989+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:13.121 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:13.117+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:13.166 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:13.162+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:13.210 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:13.207+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:13.770 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:13.764+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:13.804 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:13.798+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:13.953 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:13.948+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:13.967 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:13.962+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:14.078 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:14.073+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:14.185 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:14.180+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:14.217 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:14.211+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:14.737 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:14.732+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:14.765 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:14.760+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:14.953 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:14.948+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:15.013 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:15.008+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:15.045 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:15.040+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:15.192 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:15.187+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:15.207 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:15.203+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:15.734 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:15.729+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:15.785 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:15.780+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:15.961 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:15.956+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:16.024 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:16.019+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:16.066 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:16.060+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:16.195 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:16.189+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:16.209 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:16.204+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:16.700 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:16.695+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:16.757 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:16.751+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:16.952 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:16.948+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:17.022 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:17.016+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:17.114 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:17.109+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:17.191 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:17.187+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:17.199 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:17.194+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:17.696 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:17.691+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:17.727 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:17.722+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:17.958 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:17.953+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:18.008 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:18.004+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:18.092 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:18.087+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:18.160 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:18.156+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:18.190 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:18.185+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:18.707 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:18.703+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:18.751 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:18.747+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:19.004 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:18.999+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:19.024 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:19.020+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:19.138 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:19.132+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:19.159 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:19.155+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:19.191 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:19.187+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:19.695 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:19.690+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:19.750 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:19.745+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:20.018 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:20.014+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:20.035 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:20.030+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:20.142 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:20.137+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:20.153 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:20.148+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:20.201 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:20.196+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:20.687 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:20.681+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:20.710 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:21:20.704+0000 16949700 -1 osd.6 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:24.914466+0000 front 2019-11-25T16:20:26.695846+0000 (oldest deadline 2019-11-25T16:21:08.903063+0000) 2019-11-25T16:21:21.005 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:20.999+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:21.017 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:21.013+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:21.119 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:21.115+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:21.180 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:21.175+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:21.204 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:21.198+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:21.690 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:21.685+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:21.973 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:21.969+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:22.034 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:22.029+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:22.120 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:22.115+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:22.143 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:22.137+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:22.191 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:22.187+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:22.730 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:22.725+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:23.020 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:23.015+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:23.074 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:23.068+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:23.125 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:23.119+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:23.146 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:23.142+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:23.161 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:23.157+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:21:23.759 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:21:23.754+0000 16949700 -1 osd.5 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.554875+0000 front 2019-11-25T16:20:23.559208+0000 (oldest deadline 2019-11-25T16:21:08.754499+0000) 2019-11-25T16:21:23.977 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:21:23.972+0000 16949700 -1 osd.0 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:27.217818+0000 front 2019-11-25T16:20:25.567093+0000 (oldest deadline 2019-11-25T16:21:06.458256+0000) 2019-11-25T16:21:24.066 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:21:24.061+0000 16949700 -1 osd.7 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.394626+0000 front 2019-11-25T16:20:25.417810+0000 (oldest deadline 2019-11-25T16:21:09.440157+0000) 2019-11-25T16:21:24.117 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:21:24.112+0000 16949700 -1 osd.2 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:26.981256+0000 front 2019-11-25T16:20:26.983502+0000 (oldest deadline 2019-11-25T16:21:10.542129+0000) 2019-11-25T16:21:24.163 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:21:24.158+0000 16949700 -1 osd.3 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:23.229441+0000 front 2019-11-25T16:20:22.692229+0000 (oldest deadline 2019-11-25T16:21:07.523176+0000) 2019-11-25T16:21:24.197 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:21:24.192+0000 16949700 -1 osd.1 303 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:20:25.970574+0000 front 2019-11-25T16:20:26.702081+0000 (oldest deadline 2019-11-25T16:21:11.254114+0000) 2019-11-25T16:22:13.109 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:13.105+0000 16949700 -1 osd.3 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:14.107 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:14.104+0000 16949700 -1 osd.3 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:15.132 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:15.128+0000 16949700 -1 osd.3 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:16.121 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:16.117+0000 16949700 -1 osd.3 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:17.152 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:17.148+0000 16949700 -1 osd.3 306 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:17.608 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:22:17.549+0000 38276700 -1 osd.6 307 build_incremental_map_msg missing incremental map 307 2019-11-25T16:22:17.957 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:22:17.765+0000 36272700 -1 osd.6 307 build_incremental_map_msg missing incremental map 307 2019-11-25T16:22:18.033 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:22:17.923+0000 36272700 -1 osd.6 307 build_incremental_map_msg missing incremental map 307 2019-11-25T16:22:18.092 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:22:18.042+0000 3b27c700 -1 osd.6 307 build_incremental_map_msg missing incremental map 307 2019-11-25T16:22:18.243 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:18.239+0000 16949700 -1 osd.3 307 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:19.287 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:19.283+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:20.375 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:20.368+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:21.392 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:21.388+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:22.366 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:22.362+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:23.410 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:23.405+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:24.423 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:24.417+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:25.446 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:25.441+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:26.414 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:26.409+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:27.435 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:27.430+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:28.446 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:28.441+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:29.428 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:29.423+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:30.481 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:30.477+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:31.090 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:07:35.673 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.100 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:07:35.683 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.110 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:07:35.716 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.116 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:07:35.722 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.117 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:07:35.723 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.127 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:07:35.733 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.283 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:07:35.866 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.321 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:07:35.927 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.343 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:07:35.949 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:31.451 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:31.448+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:32.416 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:32.413+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:33.455 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:33.451+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:34.470 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:34.467+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:35.509 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:35.504+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:36.504 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:36.500+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:37.503 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:37.499+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:38.531 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:38.526+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:39.493 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:39.489+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:40.496 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:40.492+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:41.483 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:41.479+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:42.511 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:42.507+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:43.526 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:43.522+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:44.573 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:44.569+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:44.994 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:07:49.577 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:44.995 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:07:49.579 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:45.029 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:07:49.612 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:45.035 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:07:49.641 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:45.048 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:07:49.654 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:45.051 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:07:49.657 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:45.053 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:07:49.660 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:45.092 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:07:49.698 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:45.101 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:07:49.707 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:22:45.589 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:45.585+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:46.593 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:46.590+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:47.565 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:47.561+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:48.188 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:48.183+0000 16949700 -1 osd.4 308 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:115802 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 5433071200 gen 9] snapc 0=[] ondisk+write+known_if_redirected e307) 2019-11-25T16:22:48.593 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:48.586+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:49.152 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:49.147+0000 16949700 -1 osd.4 308 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:115802 5.3 5:c609908c:::notify.5:head [watch reconnect cookie 5433071200 gen 9] snapc 0=[] ondisk+write+known_if_redirected e307) 2019-11-25T16:22:49.594 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:49.591+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:50.167 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:50.162+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:50.576 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:50.571+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:51.155 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:51.151+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:51.561 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:51.557+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:52.183 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:52.179+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:52.550 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:52.546+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:53.145 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:53.141+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:53.585 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:53.582+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:54.108 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:54.103+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:54.593 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:54.588+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:55.132 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:55.127+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:55.597 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:55.593+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:56.101 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:56.096+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:56.600 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:56.597+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:57.061 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:57.055+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:57.623 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:57.619+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:58.053 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:58.048+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:58.660 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:58.656+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:22:59.057 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:22:59.052+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:22:59.641 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:22:59.637+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:00.053 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:00.048+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:00.632 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:00.628+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:01.045 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:01.040+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:01.603 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:01.600+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:02.090 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:02.085+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:02.582 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:02.578+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:03.142 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:03.137+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:03.551 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:03.548+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:04.148 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:04.143+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:04.560 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:04.555+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:05.139 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:05.134+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:05.577 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:05.572+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:06.189 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:06.184+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:06.574 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:06.571+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:07.180 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:07.175+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:07.560 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:07.556+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:08.209 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:08.204+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:08.242 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:08.238+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:08.583 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:08.578+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:08.584 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:08.579+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:08.925 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:08.919+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:09.057 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:09.052+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:09.204 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:09.199+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:09.232 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:09.227+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:09.632 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:09.627+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:09.633 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:09.628+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:09.934 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:09.929+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:10.105 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:10.100+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:10.205 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:10.201+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:10.212 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:10.205+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:10.598 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:10.593+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:10.598 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:10.594+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:10.902 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:10.897+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:11.007 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:11.003+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:11.096 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:11.092+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:11.208 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:11.203+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:11.240 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:11.236+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:11.342 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:11.337+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:11.652 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:11.646+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:11.653 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:11.648+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:11.874 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:11.869+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:12.035 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:12.030+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:12.129 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:12.124+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:12.198 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:12.194+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:12.201 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:12.197+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:12.257 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:12.252+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:12.342 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:12.338+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:12.698 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:12.693+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:12.700 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:12.697+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:12.859 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:12.854+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:13.075 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:13.071+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:13.114 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:13.109+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:13.203 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:13.198+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:13.205 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:13.201+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:13.215 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:13.211+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:13.374 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:13.369+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:13.719 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:13.713+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:13.719 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:13.715+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:13.883 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:13.878+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:14.048 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:14.043+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:14.105 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:14.100+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:14.212 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:14.207+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:14.213 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:14.208+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:14.246 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:14.242+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:14.372 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:14.366+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:14.756 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:14.752+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:14.756 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:14.752+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:14.853 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:14.845+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:15.074 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:15.069+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:15.092 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:15.087+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:15.206 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:15.201+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:15.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:15.224+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:15.232 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:15.226+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:15.405 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:15.400+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:15.794 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:15.790+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:15.794 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:15.791+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:15.858 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:15.853+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:16.090 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:16.084+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:16.129 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:16.124+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:16.196 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:16.187+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:16.258 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:16.253+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:16.260 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:16.256+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:16.433 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:16.428+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:16.781 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:16.775+0000 16949700 -1 osd.3 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.390523+0000 front 2019-11-25T16:22:25.941352+0000 (oldest deadline 2019-11-25T16:23:08.063715+0000) 2019-11-25T16:23:16.783 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:16.779+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:16.889 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:16.884+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:17.109 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:17.105+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:17.140 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:17.135+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:17.153 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:17.149+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:17.274 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:17.268+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:17.275 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:17.270+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:17.425 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:17.419+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:17.812 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:17.807+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:17.862 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:17.857+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:18.076 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:18.070+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:18.109 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:18.103+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:18.196 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:18.191+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:18.237 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:18.231+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:18.238 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:18.233+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:18.469 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:18.465+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:18.833 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:18.827+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:18.838 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:18.834+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:19.081 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:19.076+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:19.082 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:19.077+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:19.157 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:19.151+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:19.234 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:19.229+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:19.237 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:19.233+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:19.451 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:19.446+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:19.861 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:19.856+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:19.876 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:19.870+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:20.043 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:20.038+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:20.120 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:20.114+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:20.144 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:20.138+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:20.223 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:20.218+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:20.226 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:20.222+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:20.433 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:20.428+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:20.862 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:20.857+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:20.919 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:20.914+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:21.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:21.028+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:21.152 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:21.146+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:21.166 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:21.161+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:21.241 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:21.235+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:21.241 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:21.237+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:21.414 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:21.409+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:21.818 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:21.812+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:21.910 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:21.905+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:22.010 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:22.004+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:22.134 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:22.128+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:22.177 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:22.171+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:22.252 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:22.247+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:22.256 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:22.251+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:22.406 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:22.401+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:22.786 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:22.783+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:22.957 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:22.954+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:23.061 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:23.056+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:23.121 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:23.116+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:23.200 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:23.195+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:23.302 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:23.298+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:23.305 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:23.302+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:23.392 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:23.387+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:23.778 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:23.774+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:23.945 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:23.941+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:24.084 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:24.081+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:24.100 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:24.095+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:24.234 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:24.229+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:24.309 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:24.301+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:24.310 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:24.303+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:24.426 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:24.423+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:24.744 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:24.741+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:24.943 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:24.939+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:25.061 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:25.056+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:25.110 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:25.105+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:25.206 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:25.202+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:25.350 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:25.347+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:25.352 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:25.349+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:25.469 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:25.465+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:25.720 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:25.717+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:25.981 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:25.978+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:26.077 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:26.073+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:26.096 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:26.091+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:26.209 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:26.204+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:26.372 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:26.362+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:26.373 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:26.368+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:26.503 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:26.499+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:26.731 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:26.727+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:26.967 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:26.962+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:27.083 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:27.077+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:27.116 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:27.112+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:27.171 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:27.166+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:27.395 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:27.389+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:27.400 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:27.391+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:27.539 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:27.535+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:27.703 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:27.699+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:27.973 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:27.970+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:28.139 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:28.135+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:28.169 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:28.165+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:28.222 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:28.218+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:28.442 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:28.437+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:28.443 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:28.439+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:28.506 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:28.501+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:28.736 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:28.730+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:28.983 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:28.979+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:29.131 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:29.126+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:29.210 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:29.205+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:29.238 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:29.233+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:29.461 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:29.457+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:29.479 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:29.474+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:29.480 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:29.476+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:29.771 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:29.766+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:29.944 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:29.940+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:30.178 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:30.172+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:30.193 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:30.188+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:30.235 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:30.228+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:30.447 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:30.444+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:30.450 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:30.447+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:30.491 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:30.487+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:30.819 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:30.815+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:30.973 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:30.969+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:31.159 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:31.155+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:31.213 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:31.208+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:31.274 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:31.269+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:31.421 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:31.418+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:31.422 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:31.419+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:31.544 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:31.540+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:31.806 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:31.801+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:31.944 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:31.939+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:32.192 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:32.187+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:32.240 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:32.235+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:32.240 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:32.235+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:32.435 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:32.431+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:32.437 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:32.434+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:32.582 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:32.578+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:32.856 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:32.852+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:32.984 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:32.980+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:33.180 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:33.176+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:33.209 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:33.204+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:33.272 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:33.268+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:33.424 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:33.420+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:33.428 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:33.424+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:33.603 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:33.600+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:33.856 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:33.852+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:33.976 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:33.972+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:34.166 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:34.161+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:34.223 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:23:34.218+0000 16949700 -1 osd.1 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.813614+0000 front 2019-11-25T16:22:24.816394+0000 (oldest deadline 2019-11-25T16:23:08.255493+0000) 2019-11-25T16:23:34.267 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:34.262+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:22:26.547076+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:34.476 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:34.472+0000 16949700 -1 osd.4 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.463402+0000 front 2019-11-25T16:22:25.844861+0000 (oldest deadline 2019-11-25T16:23:11.505374+0000) 2019-11-25T16:23:34.479 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:23:34.476+0000 16949700 -1 osd.4 308 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e307) 2019-11-25T16:23:34.626 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:23:34.622+0000 16949700 -1 osd.5 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.926616+0000 front 2019-11-25T16:22:26.928819+0000 (oldest deadline 2019-11-25T16:23:10.389236+0000) 2019-11-25T16:23:34.876 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:34.872+0000 16949700 -1 osd.3 308 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:34.933 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:23:34.929+0000 16949700 -1 osd.7 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:24.046204+0000 front 2019-11-25T16:22:24.050345+0000 (oldest deadline 2019-11-25T16:23:08.734882+0000) 2019-11-25T16:23:35.197 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:23:35.192+0000 16949700 -1 osd.0 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:25.256784+0000 front 2019-11-25T16:22:25.265480+0000 (oldest deadline 2019-11-25T16:23:10.508065+0000) 2019-11-25T16:23:35.230 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:23:35.224+0000 16949700 -1 osd.2 308 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:22:26.389388+0000 front 2019-11-25T16:23:35.039170+0000 (oldest deadline 2019-11-25T16:23:07.280603+0000) 2019-11-25T16:23:35.246 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:23:35.232+0000 34a6f700 -1 osd.6 308 build_incremental_map_msg missing incremental map 307 2019-11-25T16:23:35.256 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:23:35.243+0000 38a77700 -1 osd.6 308 build_incremental_map_msg missing incremental map 307 2019-11-25T16:23:36.118 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:36.052+0000 16949700 -1 osd.3 309 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:23:37.314 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:23:37.308+0000 16949700 -1 osd.3 310 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115742 2.2s0 2:4100100c:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_e6937e7be00148a5bf63fab9d8b069c2:head [cmpxattr user.rgw.idtag (33) op 1 mode 1 in=47b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e305) 2019-11-25T16:24:23.229 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:23.225+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:24.244 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:24.238+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:25.292 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:25.288+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:26.305 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:26.301+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:27.305 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:27.301+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:28.320 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:28.316+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:29.342 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:29.337+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:30.307 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:30.303+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:31.341 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:31.337+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:32.369 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:32.365+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:33.348 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:33.344+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:34.321 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:34.316+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:35.337 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:35.333+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:36.352 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:36.348+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:37.386 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:37.381+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:38.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:38.350+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:39.408 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:39.404+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:40.372 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:40.368+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:41.326 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:41.322+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:41.363 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:41.360+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:42.088 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:42.082+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:42.355 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:42.351+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:42.358 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:42.352+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:42.508 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:42.502+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:43.107 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:43.101+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:43.351 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:43.348+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:43.405 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:43.401+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:43.514 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:43.509+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:43.738 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:43.732+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:44.094 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:44.089+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:44.319 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:44.314+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:44.320 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:44.316+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:44.431 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:44.427+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:44.563 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:44.557+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:44.704 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:44.699+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:45.124 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:45.118+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:45.339 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:45.333+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:45.340 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:45.334+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:45.416 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:45.409+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:45.518 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:45.514+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:45.689 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:45.684+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:45.828 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:45.823+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:45.984 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:45.978+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:46.167 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:46.161+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:46.365 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:46.360+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:46.368 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:46.364+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:46.463 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:46.458+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:46.569 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:46.563+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:46.654 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:46.650+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:46.802 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:46.797+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:47.019 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:47.012+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:47.165 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:47.161+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:47.379 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:47.374+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:47.380 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:47.376+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:47.432 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:47.428+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:47.576 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:47.570+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:47.636 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:47.630+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:47.764 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:47.760+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:48.009 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:48.003+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:48.153 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:48.148+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:48.408 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:48.403+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:48.411 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:48.406+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:48.440 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:48.435+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:48.596 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:48.590+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:48.684 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:48.680+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:48.798 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:48.792+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:48.969 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:48.965+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:49.125 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:49.120+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:49.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:49.426+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:49.432 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:49.428+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:49.446 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:49.441+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:49.566 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:49.561+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:49.667 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:49.661+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:49.843 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:49.838+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:50.019 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:50.011+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:50.116 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:50.110+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:50.437 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:50.431+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:50.438 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:50.434+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:50.484 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:50.480+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:50.599 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:50.594+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:50.657 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:50.652+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:50.881 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:50.876+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:50.983 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:50.978+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:51.075 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:51.070+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:51.438 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:51.433+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:51.439 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:51.435+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:51.458 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:51.452+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:51.571 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:51.566+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:51.636 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:51.631+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:51.896 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:51.890+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:52.025 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:52.020+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:52.087 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:52.082+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:52.422 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:52.416+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:52.423 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:52.418+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:52.472 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:52.467+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:52.582 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:52.576+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:52.612 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:52.606+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:52.895 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:52.890+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:52.984 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:52.979+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:53.096 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:53.091+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:53.441 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:53.435+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:53.442 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:53.437+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:53.497 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:53.491+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:53.541 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:53.535+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:53.577 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:53.572+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:53.924 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:53.918+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:53.974 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:53.969+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:54.112 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:54.107+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:54.449 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:54.444+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:54.451 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:54.447+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:54.510 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:54.505+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:54.516 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:54.509+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:54.583 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:54.577+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:54.941 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:54.937+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:54.943 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:54.939+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:55.131 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:55.124+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:55.486 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:55.482+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:55.488 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:55.485+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:55.502 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:55.497+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:55.560 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:55.555+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:55.569 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:55.564+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:55.912 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:55.907+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:55.944 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:55.940+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:56.179 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:56.174+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:56.481 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:56.475+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:56.509 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:56.503+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:56.510 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:56.505+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:56.536 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:56.530+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:56.565 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:56.561+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:56.919 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:56.914+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:56.975 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:56.971+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:57.174 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:57.168+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:57.487 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:57.481+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:57.487 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:57.483+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:57.489 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:57.484+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:57.493 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:57.488+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:57.606 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:57.600+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:57.889 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:57.883+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:57.982 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:57.977+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:58.141 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:58.135+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:58.467 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:58.462+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:58.469 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:58.465+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:58.524 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:58.519+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:58.538 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:58.534+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:58.565 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:58.560+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:58.896 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:58.890+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:58.976 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:58.971+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:24:59.157 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:24:59.151+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:24:59.443 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:59.437+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:24:59.444 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:24:59.440+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:24:59.527 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:24:59.522+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:24:59.536 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:24:59.531+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:24:59.564 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:24:59.559+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:24:59.921 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:24:59.915+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:24:59.959 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:24:59.953+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:25:00.119 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:25:00.113+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:25:00.482 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:00.477+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:25:00.483 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:00.479+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:00.490 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:25:00.485+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:25:00.571 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:25:00.565+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:25:00.581 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:25:00.574+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:25:00.920 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:00.915+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:25:00.936 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:25:00.932+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:25:01.144 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:25:01.138+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:25:01.483 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:25:01.478+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:25:01.518 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:01.512+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:25:01.521 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:01.516+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:01.573 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:25:01.568+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:25:01.574 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:25:01.570+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:25:01.932 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:25:01.925+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:25:01.958 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:01.952+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:25:02.131 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:25:02.125+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:25:02.509 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:02.499+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:25:02.509 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:02.501+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:02.535 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:25:02.529+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:25:02.568 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:25:02.562+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:25:02.618 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:25:02.612+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:25:02.951 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:02.943+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:25:02.982 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:25:02.976+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:25:03.177 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:25:03.173+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:25:03.544 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:03.538+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:25:03.545 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:03.540+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:03.552 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:25:03.546+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:25:03.587 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:25:03.581+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:25:03.622 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:25:03.615+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:25:03.956 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:25:03.948+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:25:03.980 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:03.975+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:25:04.186 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:25:04.181+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:25:04.537 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:04.530+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:25:04.537 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:04.532+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:04.587 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:25:04.581+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:25:04.620 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:25:04.613+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:25:04.664 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:25:04.659+0000 16949700 -1 osd.7 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.997485+0000 front 2019-11-25T16:24:00.004802+0000 (oldest deadline 2019-11-25T16:24:42.409734+0000) 2019-11-25T16:25:04.945 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:25:04.940+0000 16949700 -1 osd.2 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.415426+0000 front 2019-11-25T16:24:02.233657+0000 (oldest deadline 2019-11-25T16:24:44.916828+0000) 2019-11-25T16:25:05.023 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:05.018+0000 16949700 -1 osd.6 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.445086+0000 front 2019-11-25T16:24:02.417155+0000 (oldest deadline 2019-11-25T16:24:45.682668+0000) 2019-11-25T16:25:05.236 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:25:05.232+0000 16949700 -1 osd.5 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:59.230858+0000 front 2019-11-25T16:23:59.229911+0000 (oldest deadline 2019-11-25T16:24:41.967894+0000) 2019-11-25T16:25:05.517 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:05.510+0000 16949700 -1 osd.3 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:23:58.294364+0000 front 2019-11-25T16:23:59.103597+0000 (oldest deadline 2019-11-25T16:24:43.477222+0000) 2019-11-25T16:25:05.524 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:05.512+0000 16949700 -1 osd.3 314 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:05.575 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:25:05.568+0000 16949700 -1 osd.0 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:02.059877+0000 front 2019-11-25T16:24:01.009262+0000 (oldest deadline 2019-11-25T16:24:43.016425+0000) 2019-11-25T16:25:05.611 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:25:05.604+0000 16949700 -1 osd.1 314 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:24:01.069350+0000 front 2019-11-25T16:24:02.416504+0000 (oldest deadline 2019-11-25T16:24:41.049006+0000) 2019-11-25T16:25:06.498 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:06.494+0000 16949700 -1 osd.3 315 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:07.694 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:07.640+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:08.694 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:08.690+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:09.666 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:09.663+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:10.667 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:10.663+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:11.689 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:11.684+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:12.661 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:12.656+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:13.692 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:13.688+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:14.671 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:14.667+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:15.642 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:15.637+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:16.628 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:16.624+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:17.592 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:17.588+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:18.554 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:18.549+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:19.545 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:19.540+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:20.519 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:20.515+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:21.515 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:21.510+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:22.484 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:22.480+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:23.490 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:23.487+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:24.526 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:24.522+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:25.580 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:25.576+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:26.586 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:26.583+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:27.639 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:27.634+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:28.606 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:28.603+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:29.585 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:29.581+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:30.623 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:25:30.619+0000 16949700 -1 osd.3 316 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:115971 2.34s0 2:2e73bc6e:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.650_fde29829dc384b078744eeda1fc4d14a:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e314) 2019-11-25T16:25:36.925 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:36.710+0000 16949700 -1 osd.6 317 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected e315) 2019-11-25T16:25:38.028 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:38.005+0000 16949700 -1 osd.6 317 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected e315) 2019-11-25T16:25:39.484 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:39.107+0000 16949700 -1 osd.6 317 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected e315) 2019-11-25T16:25:40.542 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:25:40.172+0000 16949700 -1 osd.6 317 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=7 ondisk+retry+read+known_if_redirected e315) 2019-11-25T16:25:53.320 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:25:53.317+0000 39278700 -1 osd.2 318 build_incremental_map_msg missing incremental map 318 2019-11-25T16:25:53.529 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:25:53.362+0000 39278700 -1 osd.2 318 build_incremental_map_msg missing incremental map 318 2019-11-25T16:25:53.689 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:25:53.677+0000 35270700 -1 osd.1 318 build_incremental_map_msg missing incremental map 318 2019-11-25T16:25:54.044 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:25:53.873+0000 37274700 -1 osd.1 318 build_incremental_map_msg missing incremental map 318 2019-11-25T16:26:15.817 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:11:20.400 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:15.827 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:11:20.410 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:15.837 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:11:20.443 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:15.841 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:11:20.447 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:15.841 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:11:20.447 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:15.846 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:11:20.452 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:16.071 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:11:20.677 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:16.131 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:11:20.714 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:16.150 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:11:20.754 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:26:24.253 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:24.248+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:25.228 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:25.223+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:26.240 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:26.235+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:27.237 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:27.231+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:28.237 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:28.226+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:28.237 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:28.233+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:29.212 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:29.206+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:29.213 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:29.209+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:30.260 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:30.254+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:30.261 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:30.255+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:30.629 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:30.625+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:30.719 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:30.714+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:30.949 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:30.943+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:31.259 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:31.253+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:31.259 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:31.254+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:31.510 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:31.504+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:31.648 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:31.642+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:31.672 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:31.668+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:31.977 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:31.972+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:32.233 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:32.227+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:32.234 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:32.229+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:32.466 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:32.463+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:32.633 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:32.629+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:32.669 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:32.663+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:33.002 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:32.997+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:33.191 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:33.185+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:33.195 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:33.190+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:33.372 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:33.366+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:33.467 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:33.462+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:33.610 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:33.606+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:33.683 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:33.679+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:33.693 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:33.689+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:33.973 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:33.968+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:34.240 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:34.235+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:34.240 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:34.236+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:34.344 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:34.339+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:34.543 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:34.538+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:34.577 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:34.572+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:34.705 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:34.700+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:34.711 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:34.706+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:34.969 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:34.964+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:35.286 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:35.280+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:35.286 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:35.282+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:35.343 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:35.338+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:35.542 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:35.537+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:35.581 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:35.577+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:35.744 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:35.740+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:35.755 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:35.750+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:35.965 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:35.960+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:36.257 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:36.251+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:36.258 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:36.253+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:36.366 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:36.361+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:36.516 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:36.511+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:36.551 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:36.546+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:36.742 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:36.738+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:36.811 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:36.805+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:37.005 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:37.000+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:37.233 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:37.227+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:37.234 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:37.229+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:37.414 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:37.408+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:37.552 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:37.548+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:37.563 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:37.559+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:37.767 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:37.761+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:37.781 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:37.777+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:37.986 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:37.981+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:38.215 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:38.209+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:38.216 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:38.211+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:38.409 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:38.404+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:38.526 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:38.522+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:38.532 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:38.527+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:38.769 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:38.764+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:38.795 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:38.790+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:38.953 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:38.948+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:39.255 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:39.249+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:39.255 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:39.250+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:39.405 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:39.401+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:39.521 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:39.516+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:39.571 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:39.566+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:39.749 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:39.745+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:39.786 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:39.782+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:39.973 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:39.969+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:40.238 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:40.232+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:40.239 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:40.234+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:40.452 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:40.446+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:40.492 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:40.487+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:40.615 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:40.611+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:40.702 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:40.698+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:40.788 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:40.783+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:41.026 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:41.022+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:41.206 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:41.201+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:41.209 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:41.205+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:41.478 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:41.472+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:41.486 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:41.481+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:41.594 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:41.590+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:41.692 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:41.687+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:41.794 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:41.789+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:42.069 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:42.064+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:42.207 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:42.202+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:42.210 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:42.206+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:42.486 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:42.482+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:42.504 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:42.499+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:42.564 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:42.559+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:42.704 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:42.700+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:42.775 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:42.771+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:43.104 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:43.098+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:43.222 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:43.217+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:43.225 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:43.221+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:43.444 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:43.439+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:43.545 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:43.540+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:43.616 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:43.611+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:43.722 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:43.717+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:43.822 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:43.817+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:44.081 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:44.075+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:44.226 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:44.220+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:44.227 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:44.222+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:44.497 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:44.491+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:44.584 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:44.577+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:44.623 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:44.619+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:44.704 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:44.697+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:44.793 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:44.789+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:45.102 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:45.098+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:45.231 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:45.227+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:45.233 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:45.230+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:45.513 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:45.508+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:45.595 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:45.590+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:45.623 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:45.618+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:45.668 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:45.663+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:45.808 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:45.804+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:46.151 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:46.147+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:46.285 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:46.281+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:46.288 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:46.285+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:46.485 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:46.479+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:46.624 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:46.619+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:46.624 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:46.619+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:46.632 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:46.628+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:46.778 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:46.774+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:47.176 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:47.171+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:47.288 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:47.284+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:47.290 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:47.287+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:47.440 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:47.435+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:47.625 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:47.621+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:47.647 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:47.641+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:47.680 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:47.675+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:47.820 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:47.815+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:48.189 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:48.184+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:48.266 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:48.261+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:48.266 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:48.262+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:48.482 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:48.475+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:48.619 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:48.612+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:48.642 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:48.638+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:48.722 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:48.716+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:48.776 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:48.770+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:49.153 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:49.149+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:49.309 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:49.304+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:49.310 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:49.306+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:49.466 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:49.460+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:49.591 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:49.585+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:49.650 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:49.646+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:49.691 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:49.685+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:49.786 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:49.781+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:50.123 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:50.116+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:50.268 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:50.263+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:50.269 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:50.265+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:50.518 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:50.513+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:50.562 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:50.557+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:50.667 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:50.657+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:50.680 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:50.674+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:50.775 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:50.769+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:51.088 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:51.084+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:51.310 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:51.306+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:51.311 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:51.307+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:51.504 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:51.498+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:51.608 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:51.603+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:51.663 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:51.657+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:51.667 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:51.662+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:51.766 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:51.760+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:52.127 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:52.123+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:52.303 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:52.297+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:52.303 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:52.299+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:52.474 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:52.468+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:52.629 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:52.623+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:52.648 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:52.645+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:52.658 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:52.652+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:52.794 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:52.789+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:53.148 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:53.144+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:53.341 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:53.337+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:53.343 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:53.340+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:53.514 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:53.508+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:53.622 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:53.616+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:53.663 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:53.658+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:53.714 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:53.708+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:53.808 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:53.802+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:54.140 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:54.136+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:54.364 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:54.360+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:54.366 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:54.360+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:54.534 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:54.530+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:54.633 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:54.626+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:54.714 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:54.710+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:54.735 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:54.730+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:54.826 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:54.822+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:26:55.095 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:26:55.090+0000 16949700 -1 osd.7 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.771088+0000 front 2019-11-25T16:25:47.520996+0000 (oldest deadline 2019-11-25T16:26:30.854137+0000) 2019-11-25T16:26:55.323 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:55.318+0000 16949700 -1 osd.4 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:25:47.595226+0000 (oldest deadline 2019-11-25T16:26:27.595226+0000) 2019-11-25T16:26:55.323 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:26:55.319+0000 16949700 -1 osd.4 319 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e318) 2019-11-25T16:26:55.552 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:26:55.547+0000 16949700 -1 osd.0 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:46.770205+0000 front 2019-11-25T16:25:46.818330+0000 (oldest deadline 2019-11-25T16:26:30.212548+0000) 2019-11-25T16:26:55.658 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:26:55.653+0000 16949700 -1 osd.1 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:45.441056+0000 front 2019-11-25T16:25:44.775433+0000 (oldest deadline 2019-11-25T16:26:30.425081+0000) 2019-11-25T16:26:55.693 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:26:55.689+0000 16949700 -1 osd.5 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.965671+0000 front 2019-11-25T16:25:49.977271+0000 (oldest deadline 2019-11-25T16:26:32.912158+0000) 2019-11-25T16:26:55.757 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:26:55.752+0000 16949700 -1 osd.2 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:48.301352+0000 front 2019-11-25T16:25:48.314243+0000 (oldest deadline 2019-11-25T16:26:30.561824+0000) 2019-11-25T16:26:55.860 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:26:55.856+0000 16949700 -1 osd.3 319 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:25:49.318274+0000 front 2019-11-25T16:25:50.390260+0000 (oldest deadline 2019-11-25T16:26:32.894552+0000) 2019-11-25T16:28:49.895 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:49.889+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:50.353 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:50.347+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:50.884 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:50.879+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:51.405 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:51.398+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:51.870 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:51.865+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:51.891 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:51.885+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:51.911 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:51.902+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:52.146 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:28:52.139+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:28:52.395 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:52.390+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:52.419 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:28:52.415+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:28:52.864 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:52.858+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:52.881 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:52.875+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:52.885 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:52.880+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:53.198 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:28:53.193+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:28:53.398 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:53.392+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:53.443 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:28:53.437+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:28:53.843 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:53.837+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:53.931 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:53.926+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:53.938 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:53.930+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:54.164 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:28:54.159+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:28:54.352 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:54.346+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:54.431 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:28:54.426+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:28:54.828 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:54.821+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:54.904 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:54.900+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:54.920 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:28:54.915+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:28:54.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:54.974+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:55.207 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:28:55.202+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:28:55.386 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:55.380+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:55.427 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:28:55.422+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:28:55.788 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:55.783+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:55.893 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:55.887+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:55.894 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:28:55.888+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:28:56.010 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:56.005+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:56.249 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:28:56.244+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:28:56.417 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:56.412+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:56.449 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:28:56.445+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:28:56.766 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:56.760+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:56.860 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:28:56.853+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:28:56.884 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:56.879+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:57.016 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:57.011+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:57.266 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:28:57.260+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:28:57.433 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:28:57.428+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:28:57.439 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:57.435+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:57.727 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:57.721+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:57.895 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:28:57.889+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:28:57.911 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:57.907+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:57.998 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:57.990+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:58.305 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:28:58.299+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:28:58.484 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:28:58.479+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:28:58.493 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:58.488+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:58.682 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:58.677+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:58.902 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:28:58.896+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:28:58.942 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:58.937+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:58.967 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:58.961+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:28:59.331 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:28:59.325+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:28:59.445 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:28:59.438+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:28:59.524 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:28:59.520+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:28:59.735 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:28:59.729+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:28:59.868 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:28:59.863+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:28:59.913 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:28:59.909+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:28:59.925 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:28:59.919+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:00.323 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:00.318+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:00.400 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:00.395+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:00.544 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:00.539+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:00.772 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:00.767+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:00.845 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:00.840+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:00.886 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:00.881+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:00.891 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:00.886+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:01.307 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:01.302+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:01.453 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:01.448+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:01.501 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:01.497+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:01.776 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:01.770+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:01.870 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:01.865+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:01.883 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:01.878+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:01.912 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:01.906+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:02.305 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:02.300+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:02.461 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:02.455+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:02.521 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:02.512+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:02.750 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:02.744+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:02.850 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:02.846+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:02.921 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:02.915+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:02.924 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:02.919+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:03.328 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:03.322+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:03.450 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:03.445+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:03.511 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:03.507+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:03.757 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:03.750+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:03.836 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:03.832+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:03.925 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:03.920+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:03.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:03.972+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:04.363 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:04.358+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:04.477 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:04.472+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:04.552 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:04.547+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:04.756 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:04.751+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:04.825 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:04.819+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:04.892 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:04.888+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:04.947 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:04.942+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:05.322 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:05.317+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:05.439 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:05.434+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:05.512 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:05.506+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:05.749 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:05.743+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:05.835 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:05.829+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:05.920 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:05.914+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:05.925 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:05.920+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:06.324 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:06.319+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:06.432 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:06.427+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:06.519 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:06.515+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:06.790 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:06.780+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:06.803 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:06.798+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:06.893 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:06.887+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:06.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:06.961+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:07.350 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:07.344+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:07.441 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:07.436+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:07.523 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:07.518+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:07.773 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:07.768+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:07.774 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:07.768+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:07.896 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:07.890+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:07.936 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:07.931+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:08.354 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:08.349+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:08.423 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:08.418+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:08.490 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:08.485+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:08.780 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:08.776+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:08.826 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:08.821+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:08.901 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:08.897+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:08.910 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:08.905+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:09.358 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:09.352+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:09.475 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:09.470+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:09.524 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:09.519+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:09.777 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:09.772+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:09.789 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:09.785+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:09.873 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:09.869+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:09.943 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:09.937+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:10.364 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:10.359+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:10.514 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:10.510+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:10.581 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:10.576+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:10.768 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:10.763+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:10.797 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:10.792+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:10.840 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:10.834+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:10.956 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:10.949+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:11.336 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:11.330+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:11.468 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:11.464+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:11.561 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:11.556+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:11.793 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:11.788+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:11.818 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:11.812+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:11.860 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:11.855+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:11.920 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:11.915+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:12.360 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:12.355+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:12.466 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:12.461+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:12.611 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:12.606+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:12.774 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:12.769+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:12.831 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:12.826+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:12.839 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:12.832+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:12.938 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:12.932+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:13.368 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:13.362+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:13.450 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:13.445+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:13.632 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:13.627+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:13.763 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:13.758+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:13.835 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:13.829+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:13.869 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:13.864+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:13.908 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:13.902+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:14.373 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:14.368+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:14.486 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:14.482+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:14.629 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:14.623+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:14.809 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:14.803+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:14.824 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:14.820+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:14.837 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:14.830+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:14.892 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:14.886+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:15.405 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:15.400+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:15.461 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:15.457+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:15.605 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:15.599+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:15.830 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:15.824+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:15.850 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:15.845+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:15.875 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:15.869+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:15.888 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:15.882+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:16.420 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:16.414+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:16.486 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:16.481+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:16.629 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:16.623+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:16.813 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:16.808+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:16.864 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:16.859+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:16.880 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:16.875+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:16.917 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:16.913+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:17.466 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:17.461+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:17.493 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:17.487+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:17.672 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:17.666+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:17.776 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:17.771+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:17.835 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:17.830+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:17.905 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:17.901+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:17.912 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:17.907+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:18.451 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:18.446+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:18.533 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:18.528+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:18.708 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:18.694+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:18.810 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:18.805+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:18.883 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:18.878+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:18.913 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:18.908+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:18.942 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:18.937+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:19.479 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:19.475+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:19.546 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:19.541+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:19.683 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:19.678+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:19.834 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:19.830+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:19.860 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:19.856+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:19.879 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:19.875+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:19.983 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:19.979+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:20.463 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:20.458+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:20.546 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:20.541+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:20.690 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:20.685+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:20.835 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:20.829+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:20.877 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:20.873+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:20.881 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:20.877+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:20.981 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:20.973+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:21.443 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:21.440+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:21.575 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:21.571+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:21.745 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:21.740+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:21.801 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:21.795+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:21.858 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:21.854+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:21.924 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:21.920+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:21.972 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:21.967+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:22.497 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:22.492+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:22.632 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:22.627+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:22.718 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:22.714+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:22.798 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:22.792+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:22.876 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:22.872+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:22.893 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:22.885+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:23.022 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:23.015+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:23.452 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:23.448+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:23.655 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:23.649+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:23.724 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:23.718+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:23.782 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:23.778+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:23.855 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:23.850+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:23.894 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:23.890+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:24.070 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:24.065+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:24.407 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:24.404+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:24.705 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:24.700+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:24.767 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:24.762+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:24.770 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:24.765+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:24.850 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:24.845+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:24.919 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:24.915+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:25.121 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:25.117+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:25.363 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:25.359+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:25.750 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:25.745+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:29:25.782 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:29:25.776+0000 16949700 -1 osd.0 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:05.083373+0000 front 2019-11-25T16:28:05.081100+0000 (oldest deadline 2019-11-25T16:28:49.746415+0000) 2019-11-25T16:29:25.808 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:29:25.803+0000 16949700 -1 osd.3 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.510312+0000 front 2019-11-25T16:28:11.511418+0000 (oldest deadline 2019-11-25T16:28:54.869914+0000) 2019-11-25T16:29:25.851 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:29:25.847+0000 16949700 -1 osd.6 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:07.468404+0000 front 2019-11-25T16:28:07.201732+0000 (oldest deadline 2019-11-25T16:28:51.464064+0000) 2019-11-25T16:29:25.966 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:29:25.962+0000 16949700 -1 osd.7 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:09.884095+0000 front 2019-11-25T16:28:07.693733+0000 (oldest deadline 2019-11-25T16:28:49.838059+0000) 2019-11-25T16:29:26.095 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:29:26.088+0000 16949700 -1 osd.2 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.508837+0000 front 2019-11-25T16:28:11.492652+0000 (oldest deadline 2019-11-25T16:28:51.159194+0000) 2019-11-25T16:29:26.384 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:29:26.379+0000 16949700 -1 osd.5 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:08.804754+0000 front 2019-11-25T16:28:09.307269+0000 (oldest deadline 2019-11-25T16:28:51.247354+0000) 2019-11-25T16:29:26.534 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:29:26.516+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 47sec e325 v325) 2019-11-25T16:29:26.707 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:29:26.702+0000 16949700 -1 osd.1 325 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:28:11.850723+0000 front 2019-11-25T16:28:11.479827+0000 (oldest deadline 2019-11-25T16:28:51.593204+0000) 2019-11-25T16:30:17.930 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:15:22.513 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:17.939 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:15:22.522 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:17.953 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:15:22.559 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:17.960 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:15:22.566 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:17.963 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:15:22.569 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:17.973 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:15:22.579 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:18.183 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:15:22.766 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:18.236 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:15:22.842 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:18.281 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:15:22.887 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:30:35.247 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:35.242+0000 16949700 -1 osd.4 331 get_health_metrics reporting 2 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:36.273 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:36.268+0000 16949700 -1 osd.4 331 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:37.297 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:37.292+0000 16949700 -1 osd.4 331 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:38.269 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:38.265+0000 16949700 -1 osd.4 331 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:39.285 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:39.280+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:40.253 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:40.249+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:41.251 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:41.246+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:42.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:42.225+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:43.192 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:43.186+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:44.159 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:44.154+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:45.170 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:45.165+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:46.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:46.192+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:47.196 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:47.191+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:48.171 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:48.166+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:49.171 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:49.166+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:50.173 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:50.168+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:51.175 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:51.170+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:52.178 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:52.174+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:53.199 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:53.194+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:54.162 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:54.157+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:55.136 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:55.130+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:56.112 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:56.106+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:57.114 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:57.108+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:58.136 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:58.132+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:30:59.128 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:30:59.125+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:00.173 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:00.170+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:01.196 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:01.192+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:02.206 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:02.203+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:03.188 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:03.185+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:04.148 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:04.142+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:05.126 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:05.122+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:06.086 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:06.079+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:06.086 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:06.082+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:07.135 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:07.130+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:07.136 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:07.132+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:08.176 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:08.172+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:08.180 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:08.176+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:08.211 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:08.207+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:08.344 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:08.339+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:08.386 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:08.380+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:09.134 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:09.129+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:09.135 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:09.131+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:09.227 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:09.223+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:09.379 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:09.374+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:09.414 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:09.408+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:09.559 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:09.554+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:09.692 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:09.688+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:10.087 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:10.084+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:10.090 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:10.087+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:10.218 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:10.212+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:10.408 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:10.403+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:10.419 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:10.415+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:10.493 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:10.488+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:10.573 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:10.568+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:10.661 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:10.656+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:11.103 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:11.097+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:11.103 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:11.099+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:11.226 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:11.221+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:11.388 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:11.383+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:11.444 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:11.439+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:11.476 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:11.470+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:11.605 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:11.601+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:11.683 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:11.678+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:12.007 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:16:16.590 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.053 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:16:16.636 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.058 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:16:16.663 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.060 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:16:16.666 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.066 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:16:16.672 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.082 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:16:16.689 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.133 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:16:16.739 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.147 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:12.142+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:12.147 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:16:16.753 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.150 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:12.146+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:12.176 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:16:16.759 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:31:12.276 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:12.271+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:12.358 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:12.354+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:12.429 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:12.424+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:12.481 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:12.477+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:12.656 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:12.651+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:12.674 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:12.668+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:13.133 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:13.127+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:13.134 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:13.129+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:13.278 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:13.272+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:13.357 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:13.351+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:13.423 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:13.417+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:13.485 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:13.480+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:13.671 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:13.666+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:13.683 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:13.678+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:14.117 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:14.111+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:14.119 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:14.112+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:14.267 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:14.262+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:14.378 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:14.372+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:14.411 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:14.405+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:14.443 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:14.438+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:14.700 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:14.693+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:14.703 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:14.697+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:15.138 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:15.133+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:15.139 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:15.134+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:15.242 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:15.237+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:15.385 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:15.378+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:15.387 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:15.382+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:15.417 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:15.411+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:15.660 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:15.655+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:15.719 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:15.713+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:16.143 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:16.138+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:16.143 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:16.139+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:16.298 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:16.292+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:16.347 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:16.343+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:16.423 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:16.417+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:16.425 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:16.420+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:16.664 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:16.658+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:16.760 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:16.752+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:17.152 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:17.147+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:17.154 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:17.150+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:17.267 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:17.261+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:17.394 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:17.388+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:17.461 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:17.455+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:17.474 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:17.469+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:17.709 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:17.703+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:17.775 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:17.769+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:18.120 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:18.113+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:18.121 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:18.116+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:18.238 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:18.232+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:18.377 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:18.371+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:18.475 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:18.470+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:18.479 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:18.473+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:18.674 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:18.669+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:18.789 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:18.783+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:19.125 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:19.119+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:19.126 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:19.121+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:19.205 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:19.200+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:19.349 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:19.345+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:19.448 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:19.443+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:19.527 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:19.521+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:19.682 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:19.677+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:19.748 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:19.743+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:20.088 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:20.082+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:20.089 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:20.084+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:20.190 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:20.186+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:20.379 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:20.375+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:20.450 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:20.446+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:20.532 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:20.526+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:20.730 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:20.724+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:20.800 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:20.796+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:21.048 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:21.042+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:21.049 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:21.044+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:21.155 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:21.151+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:21.414 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:21.410+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:21.432 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:21.426+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:21.521 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:21.516+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:21.762 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:21.757+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:21.842 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:21.837+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:22.063 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:22.058+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:22.086 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:22.064+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:22.207 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:22.203+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:22.418 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:22.413+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:22.428 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:22.423+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:22.487 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:22.481+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:22.744 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:22.738+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:22.883 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:22.878+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:23.075 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:23.070+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:23.077 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:23.073+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:23.229 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:23.224+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:23.458 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:23.452+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:23.460 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:23.455+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:23.537 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:23.531+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:23.717 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:23.711+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:23.847 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:23.842+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:24.125 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:24.120+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:24.129 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:24.124+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:24.187 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:24.182+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:24.453 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:24.449+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:24.508 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:24.503+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:24.544 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:24.538+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:24.714 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:24.709+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:24.896 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:24.892+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:25.166 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:25.161+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:25.168 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:25.164+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:25.213 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:25.209+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:25.461 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:25.456+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:25.523 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:25.518+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:25.530 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:25.526+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:25.711 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:25.706+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:25.887 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:25.882+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:26.188 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:26.183+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:26.189 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:26.183+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:26.245 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:26.241+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:26.437 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:26.432+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:26.549 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:26.543+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:26.564 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:26.560+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:26.732 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:26.726+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:26.875 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:26.869+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:27.214 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:27.209+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:27.223 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:27.219+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:27.225 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:27.221+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:27.487 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:27.479+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:27.589 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:27.585+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:27.590 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:27.585+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:27.696 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:27.691+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:27.879 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:27.875+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:28.184 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:28.178+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:28.185 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:28.180+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:28.187 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:28.183+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:28.497 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:28.493+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:28.576 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:28.571+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:28.584 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:28.579+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:28.711 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:28.707+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:28.845 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:28.839+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:29.194 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:29.190+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:29.211 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:29.205+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:29.212 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:29.207+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:29.463 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:29.458+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:29.613 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:29.609+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:29.632 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:29.627+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:29.736 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:29.731+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:29.819 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:29.814+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:30.227 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:30.221+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:30.228 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:30.223+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:30.237 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:30.232+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:30.489 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:30.484+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:30.606 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:30.601+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:30.611 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:30.606+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:30.788 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:30.782+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:30.832 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:30.827+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:31.265 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:31.260+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:31.270 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:31.263+0000 16949700 -1 osd.4 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:29.127117+0000 front 2019-11-25T16:30:21.056833+0000 (oldest deadline 2019-11-25T16:31:05.409414+0000) 2019-11-25T16:31:31.270 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:31.266+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:31.479 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:31.475+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:31.589 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:31.584+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:31.655 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:31.650+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:31.763 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:31.759+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:31.867 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:31.863+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:32.254 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:32.251+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:32.317 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:32.313+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:32.459 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:32.455+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:32.558 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:32.552+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:32.691 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:32.687+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:32.751 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:32.747+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:32.877 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:32.872+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:33.262 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:33.257+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:33.329 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:33.324+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:33.484 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:33.479+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:33.609 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:33.604+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:33.723 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:33.719+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:33.729 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:33.724+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:33.854 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:33.850+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:34.219 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:34.214+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:34.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:34.380+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:34.465 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:34.460+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:34.659 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:34.654+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:34.719 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:34.714+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:34.751 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:34.746+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:34.901 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:34.896+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:35.177 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:35.173+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:35.426 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:35.421+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:35.468 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:35.464+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:35.669 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:35.664+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:35.752 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:35.748+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:35.778 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:35.774+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:35.909 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:35.904+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:36.209 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:36.205+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:36.451 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:36.446+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:36.479 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:36.474+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:36.656 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:36.651+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:36.744 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:36.738+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:36.829 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:36.825+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:36.899 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:36.894+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:37.072 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:31:37.067+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e331 v331) 2019-11-25T16:31:37.173 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:37.168+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:37.415 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:37.410+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:37.448 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:37.442+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:37.681 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:31:37.677+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:26.530128+0000 front 2019-11-25T16:30:26.528112+0000 (oldest deadline 2019-11-25T16:31:10.470030+0000) 2019-11-25T16:31:37.777 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:31:37.772+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.381917+0000 front 2019-11-25T16:30:29.127688+0000 (oldest deadline 2019-11-25T16:31:08.810230+0000) 2019-11-25T16:31:37.808 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:31:37.803+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.193324+0000 front 2019-11-25T16:30:28.199001+0000 (oldest deadline 2019-11-25T16:31:08.161698+0000) 2019-11-25T16:31:37.949 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:37.945+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:38.185 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:31:38.178+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:28.826377+0000 front 2019-11-25T16:30:25.595801+0000 (oldest deadline 2019-11-25T16:31:07.543359+0000) 2019-11-25T16:31:38.194 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:31:38.142+0000 34a6f700 -1 osd.6 331 build_incremental_map_msg missing incremental map 329 2019-11-25T16:31:38.259 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:31:38.246+0000 34a6f700 -1 osd.6 331 build_incremental_map_msg missing incremental map 329 2019-11-25T16:31:38.444 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:38.404+0000 16949700 -1 osd.4 331 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116316 7.a 7.d2fb1cfa (undecoded) ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:38.499 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:31:38.494+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:24.800347+0000 front 2019-11-25T16:30:25.369351+0000 (oldest deadline 2019-11-25T16:31:08.163513+0000) 2019-11-25T16:31:38.944 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:38.939+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:39.950 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:39.945+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:40.057 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:39.778+0000 16949700 -1 osd.4 331 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=11 ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:40.966 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:40.962+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:41.852 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:31:41.712+0000 16949700 -1 osd.4 331 get_health_metrics reporting 3 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=11 ondisk+retry+read+known_if_redirected e329) 2019-11-25T16:31:41.942 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:31:41.938+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:30:25.434249+0000 front 2019-11-25T16:30:25.440001+0000 (oldest deadline 2019-11-25T16:31:08.878229+0000) 2019-11-25T16:31:42.084 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:31:42.079+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 44sec e331 v331) 2019-11-25T16:32:35.014 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:35.010+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:35.203 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:35.196+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:35.876 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:35.870+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:36.027 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:36.021+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:36.229 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:36.224+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:36.547 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:36.541+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:36.899 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:36.893+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:36.990 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:36.985+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:37.214 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:37.209+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:37.532 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:37.526+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:37.773 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:37.766+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:37.901 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:37.893+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:37.982 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:37.976+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:38.186 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:38.180+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:38.237 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:38.232+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:38.491 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:38.485+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:38.798 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:38.793+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:38.919 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:38.914+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:38.943 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:38.937+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:39.213 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:39.208+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:39.289 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:39.283+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:39.360 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:39.355+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:39.460 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:39.455+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:39.823 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:39.817+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:39.951 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:39.946+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:39.972 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:39.968+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:40.180 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:40.176+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:40.331 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:40.324+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:40.392 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:40.387+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:40.479 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:40.474+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:40.871 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:40.866+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:40.950 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:40.945+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:41.001 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:40.996+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:41.153 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:41.148+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:41.352 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:41.347+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:41.399 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:41.394+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:41.530 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:41.525+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:41.838 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:41.833+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:41.968 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:41.963+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:42.009 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:42.004+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:42.169 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:42.165+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:42.367 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:42.362+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:42.446 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:42.442+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:42.564 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:42.559+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:42.822 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:42.817+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:42.964 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:42.953+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:42.984 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:42.980+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:43.212 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:43.208+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:43.408 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:43.402+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:43.451 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:43.446+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:43.523 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:43.519+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:43.784 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:43.779+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:43.996 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:43.991+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:44.025 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:44.020+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:44.211 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:44.207+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:44.433 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:44.427+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:44.486 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:44.481+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:44.499 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:44.495+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:44.794 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:44.788+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:44.973 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:44.968+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:45.024 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:45.019+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:45.201 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:45.197+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:45.452 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:45.447+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:45.472 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:45.467+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:45.484 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:45.478+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:45.794 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:45.789+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:46.012 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:46.006+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:46.068 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:46.064+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:46.255 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:46.251+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:46.442 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:46.437+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:46.496 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:46.491+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:46.504 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:46.499+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:46.830 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:46.826+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:47.008 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:47.003+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:47.170 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:47.146+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:47.282 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:47.274+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:47.441 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:47.437+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:47.480 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:47.475+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:47.551 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:47.546+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:47.848 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:47.843+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:48.026 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:48.021+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:48.111 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:48.106+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:48.288 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:48.284+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:48.442 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:48.437+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:48.523 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:48.519+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:48.524 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:48.518+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:48.809 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:48.803+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:49.059 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:49.055+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:49.140 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:49.135+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:49.300 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:49.296+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:49.480 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:49.474+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:49.482 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:49.477+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:49.564 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:49.559+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:49.791 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:49.786+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:50.090 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:50.086+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:50.139 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:50.134+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:50.346 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:50.341+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:50.463 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:50.459+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:50.504 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:50.499+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:50.587 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:50.583+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:50.791 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:50.785+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:51.135 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:51.130+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:51.193 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:51.187+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:51.325 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:51.320+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:51.421 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:51.416+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:51.479 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:51.474+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:51.599 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:51.594+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:51.760 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:51.755+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:52.176 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:52.172+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:52.185 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:52.181+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:52.324 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:52.319+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:52.426 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:52.422+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:52.498 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:52.493+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:52.574 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:52.568+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:52.756 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:52.751+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:53.179 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:53.174+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:53.229 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:53.223+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:53.323 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:53.319+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:53.382 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:53.378+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:53.486 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:53.480+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:53.607 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:53.602+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:53.724 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:53.718+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:54.168 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:54.162+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:54.262 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:54.256+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:54.369 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:54.363+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:54.408 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:54.403+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:54.480 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:54.475+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:54.658 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:54.653+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:54.682 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:54.676+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:55.156 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:55.151+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:55.272 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:55.267+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:55.369 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:55.363+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:55.388 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:55.383+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:55.457 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:55.451+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:55.667 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:55.661+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:55.734 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:55.729+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:56.166 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:56.161+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:56.303 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:56.298+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:56.412 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:56.406+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:56.420 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:56.413+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:56.505 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:56.500+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:56.664 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:56.659+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:56.693 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:56.688+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:57.207 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:57.202+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:57.272 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:57.268+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:57.450 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:57.444+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:57.462 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:57.457+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:57.552 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:57.547+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:57.628 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:57.623+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:57.653 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:57.648+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:58.223 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:58.219+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:58.255 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:58.250+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:58.473 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:58.469+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:58.503 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:58.498+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:58.537 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:58.532+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:58.619 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:58.614+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:58.626 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:58.621+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:32:59.203 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:32:59.197+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:32:59.281 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:32:59.275+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:32:59.462 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:32:59.456+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:32:59.528 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:32:59.522+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:32:59.583 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:32:59.577+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:32:59.614 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:32:59.608+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:32:59.632 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:32:59.626+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:00.241 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:00.235+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:00.248 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:00.243+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:00.438 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:00.432+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:00.570 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:00.565+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:00.581 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:00.576+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:00.594 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:00.588+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:00.653 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:00.648+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:01.257 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:01.248+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:01.258 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:01.252+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:01.416 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:01.410+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:01.565 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:01.560+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:01.567 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:01.561+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:01.583 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:01.577+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:01.649 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:01.642+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:02.216 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:02.211+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:02.240 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:02.234+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:02.399 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:02.392+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:02.524 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:02.519+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:02.574 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:02.569+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:02.580 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:02.575+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:02.686 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:02.680+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:03.187 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:03.181+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:03.226 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:03.221+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:03.389 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:03.384+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:03.509 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:03.504+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:03.564 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:03.559+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:03.576 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:03.571+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:03.719 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:03.713+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:04.209 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:04.203+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:04.270 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:04.263+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:04.348 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:04.341+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:04.471 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:04.466+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:04.589 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:04.584+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:04.589 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:04.573+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:04.752 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:04.746+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:05.255 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:05.249+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:05.287 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:05.281+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:05.333 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:05.326+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:05.518 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:05.512+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:05.573 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:05.567+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:05.622 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:05.617+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:05.760 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:05.754+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:06.306 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:06.301+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:06.324 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:06.319+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:06.333 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:06.329+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:06.509 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:06.503+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:06.571 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:06.562+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:06.633 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:06.629+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:06.812 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:06.806+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:07.324 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:07.319+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:07.331 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:07.326+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:07.373 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:33:07.368+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 43sec e331 v331) 2019-11-25T16:33:07.374 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:07.368+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:07.482 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:07.477+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:07.604 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:07.599+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:07.670 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:07.665+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:07.856 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:07.851+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:08.331 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:08.325+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:08.347 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:08.342+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:08.381 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:08.376+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:08.511 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:08.506+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:08.606 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:08.602+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:08.665 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:08.660+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:08.909 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:08.903+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:09.309 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:09.304+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:09.376 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:09.371+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:09.425 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:09.420+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:09.514 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:09.509+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:09.623 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:09.618+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:09.667 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:09.661+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:09.926 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:09.920+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:10.330 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:10.323+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:10.369 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:10.364+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:10.389 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:10.384+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:10.473 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:10.468+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:10.586 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:10.580+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:10.708 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:10.702+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:10.901 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:10.896+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:11.318 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:11.313+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:11.407 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:11.403+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:11.443 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:11.437+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:11.518 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:11.511+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:11.614 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:11.609+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:11.727 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:11.722+0000 16949700 -1 osd.6 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.306806+0000 front 2019-11-25T16:31:52.976291+0000 (oldest deadline 2019-11-25T16:32:35.648839+0000) 2019-11-25T16:33:11.872 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:33:11.867+0000 16949700 -1 osd.2 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.246761+0000 front 2019-11-25T16:31:54.496156+0000 (oldest deadline 2019-11-25T16:32:39.233019+0000) 2019-11-25T16:33:12.302 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:33:12.297+0000 16949700 -1 osd.5 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.925284+0000 front 2019-11-25T16:31:52.320984+0000 (oldest deadline 2019-11-25T16:32:34.883120+0000) 2019-11-25T16:33:12.372 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:33:12.368+0000 16949700 -1 osd.3 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.974421+0000 front 2019-11-25T16:31:52.985735+0000 (oldest deadline 2019-11-25T16:32:37.505397+0000) 2019-11-25T16:33:12.423 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:33:12.418+0000 16949700 -1 osd.1 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.179303+0000 front 2019-11-25T16:31:54.006063+0000 (oldest deadline 2019-11-25T16:32:35.515634+0000) 2019-11-25T16:33:12.517 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:33:12.513+0000 16949700 -1 osd.7 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:52.999127+0000 front 2019-11-25T16:31:52.324697+0000 (oldest deadline 2019-11-25T16:32:35.156451+0000) 2019-11-25T16:33:12.611 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:33:12.606+0000 16949700 -1 osd.0 331 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:31:54.180963+0000 front 2019-11-25T16:31:54.177560+0000 (oldest deadline 2019-11-25T16:32:36.999085+0000) 2019-11-25T16:33:12.645 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:33:12.641+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 43sec e331 v331) 2019-11-25T16:33:12.898 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:18:17.481 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:12.905 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:18:17.488 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:12.925 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:18:17.531 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:12.941 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:18:17.547 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:12.981 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:18:17.587 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:12.995 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:18:17.600 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:13.135 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:18:17.717 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:13.203 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:18:17.809 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:13.243 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:18:17.849 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:33:43.307 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:43.303+0000 16949700 -1 osd.6 333 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4500.0:116817 7.a 7:5f38df4b:::pubsub.user.foo.client.0.bucket.a868622ce869402eb73237309f960fa6%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.651:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e332) 2019-11-25T16:33:44.267 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:44.262+0000 16949700 -1 osd.6 333 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116817 7.a 7:5f38df4b:::pubsub.user.foo.client.0.bucket.a868622ce869402eb73237309f960fa6%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.651:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e332) 2019-11-25T16:33:45.251 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:45.246+0000 16949700 -1 osd.6 333 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116817 7.a 7:5f38df4b:::pubsub.user.foo.client.0.bucket.a868622ce869402eb73237309f960fa6%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.651:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e332) 2019-11-25T16:33:46.471 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:46.423+0000 16949700 -1 osd.6 334 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116817 7.a 7:5f38df4b:::pubsub.user.foo.client.0.bucket.a868622ce869402eb73237309f960fa6%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.651:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e332) 2019-11-25T16:33:47.818 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:47.814+0000 16949700 -1 osd.6 334 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116817 7.a 7:5f38df4b:::pubsub.user.foo.client.0.bucket.a868622ce869402eb73237309f960fa6%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.651:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e332) 2019-11-25T16:33:48.855 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:33:48.820+0000 16949700 -1 osd.6 335 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4500.0:116817 7.a 7:5f38df4b:::pubsub.user.foo.client.0.bucket.a868622ce869402eb73237309f960fa6%2f103c5a94-eb69-4977-8c90-9f6d634d8761.4358.651:head [getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e332) 2019-11-25T16:34:20.581 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:20.573+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:21.331 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:19:25.914 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.351 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:19:25.934 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.352 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:19:25.956 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.352 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:19:25.957 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.353 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:19:25.959 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.358 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:19:25.963 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.453 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:19:26.059 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.458 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:19:26.064 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.458 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:19:26.041 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:34:21.538 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:21.534+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:22.536 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:22.531+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:23.580 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:23.576+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:24.558 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:24.554+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:25.546 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:25.542+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:26.556 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:26.552+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:27.572 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:27.568+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:28.558 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:28.554+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:29.586 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:29.582+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:30.606 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:30.602+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:31.589 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:31.585+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:32.638 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:32.634+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:33.691 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:33.687+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:34.703 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:34.698+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:35.662 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:35.657+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:36.679 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:36.674+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:37.636 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:37.631+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:38.634 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:38.629+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:39.652 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:39.647+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:40.607 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:40.602+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:41.581 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:41.576+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:41.642 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:41.637+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:41.727 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:41.722+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:42.629 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:42.624+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:42.634 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:42.629+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:42.733 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:42.727+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:43.230 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:43.225+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:43.607 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:43.602+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:43.654 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:43.649+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:43.654 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:43.650+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:43.771 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:43.766+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:44.201 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:44.196+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:44.633 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:44.627+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:44.634 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:44.628+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:44.636 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:44.631+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:44.735 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:44.730+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:45.038 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:45.033+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:45.156 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:45.151+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:45.502 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:45.496+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:45.612 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:45.607+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:45.614 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:45.610+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:45.617 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:45.611+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:45.698 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:45.689+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:46.001 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:45.995+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:46.166 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:46.161+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:46.191 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:46.185+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:46.458 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:46.452+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:46.588 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:46.582+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:46.628 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:46.622+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:46.629 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:46.624+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:46.657 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:46.651+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:47.018 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:47.013+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:47.133 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:47.127+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:47.167 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:47.161+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:47.456 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:47.451+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:47.603 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:47.597+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:47.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:47.599+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:47.626 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:47.621+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:47.704 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:47.697+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:48.018 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:48.011+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:48.166 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:48.160+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:48.177 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:48.171+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:48.503 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:48.499+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:48.586 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:48.579+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:48.586 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:48.582+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:48.624 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:48.619+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:48.659 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:48.653+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:49.071 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:49.066+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:49.140 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:49.131+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:49.157 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:49.152+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:49.555 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:49.548+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:49.597 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:49.592+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:49.598 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:49.593+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:49.623 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:49.618+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:49.682 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:49.677+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:50.084 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:50.079+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:50.127 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:50.121+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:50.154 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:50.148+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:50.601 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:50.596+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:50.634 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:50.629+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:50.635 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:50.630+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:50.636 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:50.631+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:50.645 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:50.640+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:51.098 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:51.091+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:51.117 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:51.112+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:51.151 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:51.146+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:51.622 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:51.617+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:51.639 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:51.634+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:51.644 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:51.638+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:51.647 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:51.642+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:51.673 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:51.668+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:52.081 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:52.075+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:52.113 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:52.108+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:52.126 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:52.120+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:52.657 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:52.650+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:52.657 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:52.652+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:52.671 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:52.664+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:52.683 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:52.677+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:52.702 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:52.697+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:53.036 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:53.030+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:53.081 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:53.077+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:53.138 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:53.133+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:53.633 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:53.627+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:53.633 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:53.628+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:53.639 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:53.634+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:53.662 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:53.658+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:53.708 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:53.698+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:54.049 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:54.044+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:54.118 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:54.113+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:54.119 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:54.115+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:54.602 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:54.597+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:54.603 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:54.598+0000 16949700 -1 osd.4 337 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:54.666 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:54.661+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:54.673 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:54.668+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:54.738 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:54.734+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:55.078 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:55.073+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:55.084 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:55.080+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:55.140 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:55.134+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:55.601 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:55.597+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:55.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:55.599+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:55.647 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:55.642+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:55.686 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:55.682+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:55.737 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:55.733+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:56.055 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:56.050+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:56.099 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:56.093+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:56.165 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:56.159+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:56.583 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:56.577+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:56.586 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:56.581+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:56.620 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:56.614+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:56.721 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:56.716+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:56.749 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:56.744+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:57.088 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:57.083+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:57.099 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:57.095+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:57.189 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:57.185+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:57.540 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:57.534+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:57.542 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:57.537+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:57.666 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:57.661+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:57.707 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:57.702+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:57.775 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:57.770+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:58.084 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:58.078+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:58.126 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:58.121+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:58.203 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:58.199+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:58.600 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:58.595+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:58.600 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:58.596+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:58.688 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:58.683+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:58.761 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:58.755+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:58.801 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:58.796+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:34:59.105 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:34:59.099+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:34:59.152 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:34:59.146+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:34:59.220 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:34:59.215+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:34:59.639 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:59.634+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:34:59.640 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:34:59.635+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:34:59.707 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:34:59.702+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:34:59.739 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:34:59.732+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:34:59.798 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:34:59.793+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:00.064 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:00.059+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:00.186 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:00.178+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:00.193 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:00.188+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:00.617 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:00.612+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:00.618 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:00.613+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:00.716 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:00.711+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:00.740 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:00.735+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:00.820 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:00.815+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:01.049 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:01.044+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:01.173 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:01.168+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:01.195 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:01.191+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:01.628 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:01.623+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:01.631 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:01.626+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:01.718 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:01.713+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:01.733 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:01.728+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:01.860 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:01.854+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:02.062 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:02.057+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:02.150 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:02.146+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:02.233 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:02.228+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:02.608 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:02.603+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:02.612 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:02.607+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:02.762 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:02.757+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:02.780 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:02.774+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:02.863 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:02.858+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:03.053 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:03.049+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:03.143 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:03.138+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:03.246 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:03.240+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:03.590 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:03.583+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:03.591 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:03.585+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:03.744 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:03.738+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:03.800 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:03.795+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:03.852 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:03.847+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:04.039 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:04.033+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:04.136 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:04.131+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:04.243 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:04.237+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:04.582 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:04.575+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:04.582 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:04.577+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:04.732 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:04.727+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:04.822 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:04.816+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:04.824 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:04.819+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:05.029 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:05.023+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:05.130 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:05.125+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:05.262 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:05.256+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:05.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:05.597+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:05.605 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:05.600+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:05.709 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:05.704+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:05.782 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:05.778+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:05.827 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:05.823+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:05.989 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:05.984+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:06.169 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:06.164+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:06.242 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:06.238+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:06.589 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:06.583+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:06.590 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:06.585+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:06.734 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:06.729+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:06.774 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:06.769+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:06.784 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:06.780+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:06.977 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:06.971+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:07.200 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:07.195+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:07.207 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:07.203+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:07.562 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:07.555+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:07.563 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:07.557+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:07.698 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:07.692+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:07.781 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:07.776+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:07.835 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:07.831+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:07.950 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:07.945+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:08.178 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:08.173+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:08.234 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:08.229+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:08.545 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:08.541+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:08.548 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:08.544+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:08.732 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:08.728+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:08.776 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:08.771+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:08.837 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:08.832+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:08.911 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:08.906+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:09.138 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:09.133+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:09.239 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:09.234+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:09.589 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:09.583+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:09.590 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:09.585+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:09.729 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:09.724+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:09.813 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:09.808+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:09.814 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:09.809+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:09.951 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:09.947+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:10.165 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:10.160+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:10.269 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:10.264+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:10.582 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:10.576+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:10.582 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:10.577+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:10.733 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:10.728+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:10.842 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:10.831+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:10.865 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:10.860+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:10.906 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:10.901+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:11.130 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:11.125+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:11.296 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:11.291+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:11.547 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:11.542+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:11.548 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:11.543+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:11.792 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:11.787+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:11.847 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:11.843+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:11.873 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:11.868+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:11.923 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:11.919+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:12.111 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:12.106+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:12.284 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:12.279+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:12.524 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:12.520+0000 16949700 -1 osd.4 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.745497+0000 front 2019-11-25T16:33:58.663866+0000 (oldest deadline 2019-11-25T16:34:43.190167+0000) 2019-11-25T16:35:12.528 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:35:12.524+0000 16949700 -1 osd.4 337 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e335) 2019-11-25T16:35:12.753 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:35:12.748+0000 16949700 -1 osd.7 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.369167+0000 front 2019-11-25T16:34:00.424737+0000 (oldest deadline 2019-11-25T16:34:44.987935+0000) 2019-11-25T16:35:12.837 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:35:12.832+0000 16949700 -1 osd.3 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:59.564307+0000 front 2019-11-25T16:34:00.995725+0000 (oldest deadline 2019-11-25T16:34:40.855807+0000) 2019-11-25T16:35:12.873 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:35:12.869+0000 16949700 -1 osd.2 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:56.234039+0000 front 2019-11-25T16:33:56.335261+0000 (oldest deadline 2019-11-25T16:34:40.869758+0000) 2019-11-25T16:35:12.945 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:35:12.940+0000 16949700 -1 osd.5 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.064008+0000 front 2019-11-25T16:33:58.067946+0000 (oldest deadline 2019-11-25T16:34:42.509911+0000) 2019-11-25T16:35:13.115 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:35:13.111+0000 16949700 -1 osd.0 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:33:58.335686+0000 front 2019-11-25T16:33:58.347075+0000 (oldest deadline 2019-11-25T16:34:44.223903+0000) 2019-11-25T16:35:13.222 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:35:13.216+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e337 v337) 2019-11-25T16:35:13.325 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:35:13.320+0000 16949700 -1 osd.1 337 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:34:00.824567+0000 front 2019-11-25T16:34:00.992768+0000 (oldest deadline 2019-11-25T16:34:46.037751+0000) 2019-11-25T16:35:57.868 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:35:57.843+0000 3426e700 -1 osd.6 342 build_incremental_map_msg missing incremental map 342 2019-11-25T16:37:00.979 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:00.974+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:01.913 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:01.908+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:02.018 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:02.014+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:02.946 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:02.942+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:03.035 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:03.029+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:03.284 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:03.278+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:03.660 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:03.654+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:03.780 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:03.775+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:03.912 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:03.907+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:04.082 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:04.077+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:04.314 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:04.308+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:04.403 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:04.397+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:04.649 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:04.644+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:04.751 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:04.746+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:04.881 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:04.877+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:05.116 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:05.111+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:05.287 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:05.281+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:05.368 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:05.362+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:05.687 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:05.682+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:05.712 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:05.707+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:05.892 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:05.886+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:06.119 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:06.114+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:06.271 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:06.265+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:06.305 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:06.300+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:06.346 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:06.338+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:06.690 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:06.683+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:06.704 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:06.699+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:06.888 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:06.882+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:07.153 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:07.148+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:07.305 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:07.300+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:07.323 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:07.318+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:07.364 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:07.358+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:07.696 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:07.691+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:07.738 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:07.733+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:07.914 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:07.909+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:08.127 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:08.122+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:08.323 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:08.317+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:08.344 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:08.337+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:08.385 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:08.380+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:08.699 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:08.693+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:08.758 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:08.752+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:08.967 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:08.962+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:09.089 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:09.084+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:09.304 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:09.299+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:09.352 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:09.346+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:09.432 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:09.427+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:09.717 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:09.711+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:09.793 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:09.788+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:10.007 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:10.001+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:10.059 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:10.053+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:10.302 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:10.297+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:10.361 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:10.356+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:10.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:10.442+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:10.710 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:10.704+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:10.805 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:10.799+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:11.016 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:11.010+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:11.056 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:11.051+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:11.308 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:11.300+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:11.412 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:11.403+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:11.433 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:11.424+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:11.711 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:11.705+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:11.763 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:11.758+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:12.022 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:12.018+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:12.042 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:12.037+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:12.301 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:12.296+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:12.381 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:12.376+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:12.462 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:12.458+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:12.749 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:12.744+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:12.758 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:12.753+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:13.033 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:13.027+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:13.070 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:13.064+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:13.308 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:13.302+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:13.351 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:13.346+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:13.451 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:13.446+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:13.732 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:13.726+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:13.805 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:13.800+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:14.019 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:14.014+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:14.079 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:14.074+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:14.318 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:14.314+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:14.375 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:14.371+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:14.487 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:14.482+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:14.778 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:14.772+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:14.783 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:14.777+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:15.026 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:15.020+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:15.078 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:15.073+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:15.328 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:15.322+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:15.404 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:15.398+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:15.499 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:15.494+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:15.734 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:15.728+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:15.756 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:15.750+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:16.043 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:16.038+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:16.079 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:16.073+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:16.294 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:16.289+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:16.447 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:16.440+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:16.514 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:16.507+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:16.746 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:16.741+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:16.772 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:16.766+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:17.067 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:17.062+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:17.075 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:17.070+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:17.268 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:17.263+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:17.411 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:17.406+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:17.480 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:17.476+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:17.705 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:17.700+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:17.732 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:17.726+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:18.076 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:18.072+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:18.117 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:18.112+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:18.243 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:18.239+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:18.461 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:18.457+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:18.520 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:18.514+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:18.715 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:18.710+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:18.716 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:18.711+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:19.108 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:19.102+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:19.145 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:19.141+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:19.294 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:19.285+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:19.500 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:19.495+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:19.501 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:19.496+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:19.721 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:19.716+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:19.724 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:19.720+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:20.133 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:20.127+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:20.138 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:20.133+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:20.311 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:20.306+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:20.467 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:20.461+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:20.501 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:20.496+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:20.703 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:20.698+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:20.722 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:20.717+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:21.177 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:21.171+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:21.177 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:21.173+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:21.300 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:21.295+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:21.473 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:21.468+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:21.497 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:21.492+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:21.689 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:21.683+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:21.693 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:21.688+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:22.159 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:22.153+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:22.185 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:22.179+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:22.299 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:22.293+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:22.518 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:22.512+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:22.539 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:22.535+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:22.649 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:22.643+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:22.657 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:22.652+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:23.175 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:23.170+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:23.201 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:23.195+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:23.258 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:23.253+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:23.488 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:23.483+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:23.514 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:23.509+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:23.635 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:23.629+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:23.657 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:23.651+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:24.219 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:24.213+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:24.252 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:24.248+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:24.297 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:24.291+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:24.510 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:24.504+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:24.525 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:24.519+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:24.606 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:24.595+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:24.677 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:24.671+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:25.229 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:25.224+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:25.264 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:25.257+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:25.307 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:25.302+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:25.468 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:25.462+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:25.482 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:25.476+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:25.626 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:25.620+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:25.720 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:25.715+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:26.251 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:26.247+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:26.267 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:26.261+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:26.359 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:26.354+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:26.446 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:26.442+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:26.510 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:26.505+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:26.647 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:26.642+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:26.723 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:26.718+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:27.218 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:27.213+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:27.241 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:27.235+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:27.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:27.347+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:27.463 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:27.458+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:27.499 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:27.494+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:27.694 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:27.688+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:27.697 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:27.689+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:28.199 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:28.192+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:28.249 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:28.243+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:28.341 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:28.334+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:28.459 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:28.453+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:28.470 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:28.463+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:28.665 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:28.659+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:28.684 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:28.679+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:29.208 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:29.203+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:29.251 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:29.245+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:29.373 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:29.367+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:29.430 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:29.425+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:29.481 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:29.475+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:29.674 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:29.669+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:29.736 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:29.730+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:30.198 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:30.192+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:30.248 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:30.241+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:30.347 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:30.341+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:30.394 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:30.388+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:30.458 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:30.451+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:30.643 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:30.637+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:30.706 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:30.700+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:31.191 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:31.185+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:31.257 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:31.252+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:31.313 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:31.308+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:31.389 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:31.383+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:31.462 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:31.457+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:31.683 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:31.678+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:31.721 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:31.715+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:32.178 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:32.173+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:32.254 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:32.249+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:32.346 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:32.341+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:32.419 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:32.414+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:32.470 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:32.465+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:32.712 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:32.706+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:32.726 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:32.720+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:33.151 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:33.145+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:33.301 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:33.295+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:33.360 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:33.355+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:33.419 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:33.415+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:33.520 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:33.515+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:33.668 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:37:33.662+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 45sec e343 v343) 2019-11-25T16:37:33.718 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:33.709+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:33.764 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:33.760+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:34.159 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:34.153+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:34.348 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:34.342+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:34.390 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:34.386+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:34.458 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:34.452+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:34.533 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:34.528+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:34.763 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:34.758+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:34.790 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:34.785+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:35.132 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:35.126+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:35.312 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:35.308+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:35.404 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:35.399+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:35.431 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:35.427+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:35.546 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:35.540+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:35.731 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:35.725+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:35.762 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:35.756+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:36.165 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:36.159+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:36.316 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:36.311+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:36.416 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:36.411+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:36.459 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:36.453+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:36.570 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:36.566+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:36.761 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:36.756+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:36.778 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:36.773+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:37.195 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:37.189+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:37.355 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:37.350+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:37.386 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:37.381+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:37.491 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:37.487+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:37.548 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:37.543+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:37.757 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:37.751+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:37.820 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:37.815+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:38.154 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:38.149+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:38.376 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:38.372+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:38.440 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:38.430+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:38.494 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:38.489+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:38.595 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:38.590+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:38.685 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:37:38.681+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 45sec e343 v343) 2019-11-25T16:37:38.726 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:38.720+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:38.782 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:38.777+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:39.189 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:39.183+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:39.338 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:39.333+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:39.451 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:39.446+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:39.465 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:39.461+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:39.631 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:39.626+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:39.721 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:39.714+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:39.782 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:39.776+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:40.157 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:40.151+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:40.308 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:40.301+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:40.409 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:40.404+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:40.488 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:40.483+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:40.662 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:40.658+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:40.741 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:40.736+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:40.776 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:40.770+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:41.183 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:41.177+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:41.334 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:41.327+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:41.445 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:41.440+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:41.534 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:41.529+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:41.694 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:41.689+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:41.759 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:41.752+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:41.794 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:41.788+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:42.152 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:42.137+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:42.363 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:42.357+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:42.453 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:42.449+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:42.586 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:42.581+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:42.717 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:42.712+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:42.797 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:42.792+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:42.845 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:42.839+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:37:43.184 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:37:43.178+0000 16949700 -1 osd.5 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.241677+0000 front 2019-11-25T16:36:22.316577+0000 (oldest deadline 2019-11-25T16:37:01.380156+0000) 2019-11-25T16:37:43.402 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:37:43.398+0000 16949700 -1 osd.1 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:17.786986+0000 front 2019-11-25T16:36:17.829342+0000 (oldest deadline 2019-11-25T16:37:00.875169+0000) 2019-11-25T16:37:43.418 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:37:43.414+0000 16949700 -1 osd.3 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:21.176591+0000 front 2019-11-25T16:36:17.313834+0000 (oldest deadline 2019-11-25T16:37:03.176428+0000) 2019-11-25T16:37:43.620 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:37:43.616+0000 16949700 -1 osd.2 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069609+0000 front 2019-11-25T16:36:22.275347+0000 (oldest deadline 2019-11-25T16:37:05.367344+0000) 2019-11-25T16:37:43.763 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:37:43.759+0000 16949700 -1 osd.0 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:20.523968+0000 front 2019-11-25T16:36:22.222284+0000 (oldest deadline 2019-11-25T16:37:04.338001+0000) 2019-11-25T16:37:43.764 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:37:43.758+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 45sec e343 v343) 2019-11-25T16:37:43.817 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:37:43.810+0000 16949700 -1 osd.7 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:22.249115+0000 front 2019-11-25T16:36:22.247993+0000 (oldest deadline 2019-11-25T16:37:03.012757+0000) 2019-11-25T16:37:43.877 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:37:43.874+0000 16949700 -1 osd.6 343 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:36:23.069225+0000 front 2019-11-25T16:36:20.510852+0000 (oldest deadline 2019-11-25T16:37:03.336294+0000) 2019-11-25T16:38:15.388 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:15.375+0000 37274700 -1 osd.4 348 build_incremental_map_msg missing incremental map 348 2019-11-25T16:38:15.672 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:15.659+0000 36a73700 -1 osd.4 348 build_incremental_map_msg missing incremental map 348 2019-11-25T16:38:16.093 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:15.836+0000 35270700 -1 osd.4 348 build_incremental_map_msg missing incremental map 348 2019-11-25T16:38:16.359 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:16.320+0000 35270700 -1 osd.4 348 build_incremental_map_msg missing incremental map 348 2019-11-25T16:38:21.394 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:23:25.977 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:21.401 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:23:25.984 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:21.410 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:23:26.015 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:21.415 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:23:26.021 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:21.415 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:23:26.021 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:21.421 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:23:26.027 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:21.629 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:23:26.212 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:21.638 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:23:26.244 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:21.657 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:23:26.262 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:38:41.240 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:41.235+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:42.237 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:42.231+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:42.483 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:42.478+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:42.990 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:42.986+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:43.248 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:43.243+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:43.458 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:43.453+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:44.002 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:43.999+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:44.114 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:44.109+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:44.286 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:44.281+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:44.494 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:44.489+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:44.973 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:44.968+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:45.064 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:45.059+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:45.137 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:45.132+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:45.248 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:45.243+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:45.472 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:45.467+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:45.576 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:45.572+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:46.015 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:46.010+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:46.068 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:46.062+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:46.085 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:46.080+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:46.149 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:46.146+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:46.267 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:46.263+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:46.497 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:46.492+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:46.621 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:46.618+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:47.038 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:47.033+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:47.040 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:47.036+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:47.099 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:47.094+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:47.112 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:47.107+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:47.243 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:47.237+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:47.495 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:47.491+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:47.597 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:47.592+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:48.045 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:48.039+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:48.047 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:48.042+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:48.077 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:48.072+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:48.109 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:48.103+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:48.295 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:48.290+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:48.528 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:48.523+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:48.588 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:48.583+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:49.043 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:49.038+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:49.059 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:49.054+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:49.093 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:49.089+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:49.163 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:49.158+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:49.250 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:49.245+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:49.526 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:49.521+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:49.562 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:49.558+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:50.002 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:49.997+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:50.034 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:50.029+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:50.074 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:50.069+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:50.124 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:50.120+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:50.272 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:50.268+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:50.524 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:50.520+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:50.541 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:50.537+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:50.983 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:50.977+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:51.011 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:51.005+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:51.091 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:51.086+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:51.128 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:51.123+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:51.313 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:51.308+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:51.559 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:51.553+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:51.594 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:51.589+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:51.964 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:51.958+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:52.001 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:51.995+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:52.114 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:52.110+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:52.116 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:52.111+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:52.283 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:52.279+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:52.549 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:52.544+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:38:52.551 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:52.547+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:52.567 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:52.562+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:52.944 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:52.938+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:52.997 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:52.991+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:53.095 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:53.090+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:53.101 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:53.097+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:53.347 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:53.338+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:53.568 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:53.564+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:53.578 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:53.573+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:38:53.580 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:53.576+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:53.904 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:53.898+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:54.026 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:54.021+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:54.119 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:54.115+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:54.132 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:54.127+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:54.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:54.348+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:54.544 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:54.538+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:54.544 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:54.539+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:38:54.545 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:54.540+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:54.916 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:54.910+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:54.992 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:54.986+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:55.115 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:55.110+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:55.121 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:55.116+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:55.313 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:55.309+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:55.509 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:55.502+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:38:55.509 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:55.503+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:55.568 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:55.563+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:55.954 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:55.948+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:55.988 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:55.982+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:56.097 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:56.093+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:56.144 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:56.140+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:56.293 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:56.288+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:56.501 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:56.497+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:38:56.503 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:56.499+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:56.578 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:56.572+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:56.975 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:56.969+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:56.985 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:56.979+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:57.094 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:57.088+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:57.120 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:57.116+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:57.288 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:57.283+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:57.476 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:57.471+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:38:57.479 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:57.475+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:57.533 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:57.528+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:57.934 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:57.928+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:58.024 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:58.018+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:58.115 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:58.111+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:58.131 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:58.127+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:58.336 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:58.332+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:58.463 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:58.457+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:38:58.464 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:58.459+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:58.507 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:58.502+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:58.983 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:58.977+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:38:59.038 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:38:59.032+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:38:59.138 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:38:59.134+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:38:59.164 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:38:59.159+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:38:59.365 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:38:59.360+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:38:59.513 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:38:59.507+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:38:59.617 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:59.531+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:38:59.619 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:38:59.613+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:38:59.991 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:38:59.986+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:00.082 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:00.077+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:00.145 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:00.141+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:00.166 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:00.161+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:00.368 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:00.363+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:00.546 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:00.542+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:00.634 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:00.629+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:00.636 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:00.632+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:00.962 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:00.957+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:01.061 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:01.056+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:01.138 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:01.133+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:01.202 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:01.197+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:01.369 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:01.363+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:01.516 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:01.511+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:01.676 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:01.671+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:01.680 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:01.675+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:02.008 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:02.003+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:02.064 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:02.058+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:02.111 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:02.106+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:02.252 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:02.248+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:02.400 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:02.392+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:02.565 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:02.560+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:02.643 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:02.636+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:02.643 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:02.638+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:03.024 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:03.019+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:03.108 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:03.102+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:03.151 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:03.144+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:03.306 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:03.295+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:03.369 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:03.364+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:03.523 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:03.519+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:03.632 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:03.625+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:03.635 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:03.627+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:03.994 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:03.989+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:04.159 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:04.153+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:04.185 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:04.180+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:04.324 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:04.319+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:04.371 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:04.366+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:04.531 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:04.527+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:04.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:04.658+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:04.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:04.659+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:04.962 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:04.957+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:05.143 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:05.136+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:05.145 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:05.140+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:05.313 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:05.308+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:05.398 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:05.393+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:05.518 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:05.513+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:05.662 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:05.657+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:05.663 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:05.659+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:05.925 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:05.919+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:06.123 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:06.117+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:06.159 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:06.154+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:06.326 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:06.322+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:06.367 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:06.361+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:06.474 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:06.469+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:06.627 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:06.621+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:06.629 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:06.622+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:06.979 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:06.973+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:07.096 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:07.088+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:07.187 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:07.183+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:07.328 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:07.323+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:07.357 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:07.352+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:07.519 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:07.514+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:07.626 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:07.621+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:07.629 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:07.624+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:07.937 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:07.931+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:08.081 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:08.076+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:08.202 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:08.198+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:08.299 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:08.295+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:08.316 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:08.312+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:08.543 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:08.538+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:08.682 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:08.675+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:08.682 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:08.677+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:08.932 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:08.926+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:09.091 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:09.086+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:09.184 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:09.180+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:09.263 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:09.257+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:09.353 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:09.348+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:09.593 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:09.588+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:09.666 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:09.659+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:09.666 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:09.661+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:09.984 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:09.978+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:10.063 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:10.057+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:10.183 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:10.179+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:10.228 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:10.223+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:10.349 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:10.344+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:10.622 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:10.617+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:10.651 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:10.645+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:10.652 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:10.647+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:10.998 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:10.994+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:11.035 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:11.029+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:11.198 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:11.192+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:11.204 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:11.198+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:11.327 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:11.321+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:11.610 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:11.604+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:11.665 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:11.660+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:11.668 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:11.664+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:12.006 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:11.998+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:12.062 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:12.056+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:12.153 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:12.148+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:12.183 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:12.177+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:12.348 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:12.342+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:12.632 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:12.626+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:12.660 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:12.655+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:12.663 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:12.658+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:13.006 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:13.000+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:13.094 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:13.088+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:13.186 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:13.180+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:13.193 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:13.188+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:13.392 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:13.387+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:13.649 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:13.643+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:13.650 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:13.644+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:13.687 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:13.681+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:13.988 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:13.984+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:14.059 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:39:14.054+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e349 v349) 2019-11-25T16:39:14.131 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:14.127+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:14.157 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:14.152+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:14.170 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:14.165+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:14.406 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:14.401+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:14.666 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:14.661+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:14.669 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:14.665+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:14.717 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:14.710+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:14.979 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:14.974+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:15.158 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:15.152+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:15.176 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:15.170+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:15.207 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:15.201+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:15.434 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:15.426+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:15.682 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:15.676+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:15.700 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:15.692+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:15.701 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:15.694+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:15.982 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:15.976+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:16.134 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:16.128+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:16.164 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:16.158+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:16.232 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:16.227+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:16.589 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:16.582+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:16.648 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:16.643+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:16.727 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:16.722+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:16.730 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:16.726+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:16.966 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:16.962+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:17.142 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:17.137+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:17.174 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:17.169+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:17.251 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:17.246+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:17.648 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:17.644+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:17.708 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:17.701+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:17.710 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:17.704+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:17.711 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:17.706+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:18.012 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:18.007+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:18.135 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:18.130+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:18.140 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:18.134+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:18.268 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:18.263+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:18.669 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:18.664+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:18.707 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:18.700+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:18.744 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:18.739+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:18.747 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:18.743+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:18.995 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:18.989+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:19.086 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:39:19.078+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e349 v349) 2019-11-25T16:39:19.139 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:19.134+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:19.187 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:19.183+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:19.311 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:19.307+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:19.627 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:19.622+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:19.686 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:19.682+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:19.704 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:19.700+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:19.706 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:19.702+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:19.987 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:19.981+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:20.141 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:20.136+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:20.177 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:20.172+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:20.273 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:20.269+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:20.616 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:20.611+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:20.729 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:20.721+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:20.753 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:20.741+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:20.754 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:20.744+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:20.955 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:20.950+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:21.133 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:21.128+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:21.156 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:21.152+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:21.235 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:21.230+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:21.611 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:21.606+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:21.727 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:21.723+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:21.754 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:21.749+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:21.754 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:21.750+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:21.920 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:21.915+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:22.152 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:22.147+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:22.157 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:22.153+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:22.208 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:22.202+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:22.630 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:22.625+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:22.694 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:22.689+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:22.742 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:22.736+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:22.745 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:22.740+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:22.930 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:22.925+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:23.165 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:23.158+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:23.174 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:23.170+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:23.199 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:23.195+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:23.660 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:23.655+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:23.704 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:23.698+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:23.704 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:23.699+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:23.706 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:23.702+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:23.927 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:23.921+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:24.095 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:39:24.091+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e349 v349) 2019-11-25T16:39:24.140 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:24.135+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:24.161 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:24.157+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:24.183 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:24.178+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:24.663 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:24.658+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:24.697 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:24.691+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:24.700 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:24.695+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:24.909 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:24.904+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:25.033 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:25.027+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:25.126 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:25.121+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:25.139 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:25.135+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:25.184 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:25.179+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:25.633 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:25.628+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:25.702 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:25.699+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:25.705 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:25.701+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:25.889 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:25.886+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:26.078 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:26.074+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:26.096 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:26.092+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:26.148 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:26.143+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:26.170 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:26.165+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:26.595 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:26.591+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:26.723 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:26.718+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:26.724 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:26.719+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:26.910 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:26.906+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:27.073 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:27.068+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:27.089 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:27.084+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:27.143 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:27.138+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:27.190 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:27.184+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:27.553 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:27.549+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:27.717 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:27.712+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:27.718 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:27.713+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:27.923 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:27.919+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:28.063 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:28.060+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:28.114 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:28.110+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:28.131 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:28.126+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:28.216 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:28.211+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:28.599 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:28.594+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:28.733 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:28.728+0000 16949700 -1 osd.4 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:38:12.198678+0000 (oldest deadline 2019-11-25T16:38:52.198678+0000) 2019-11-25T16:39:28.735 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:28.730+0000 16949700 -1 osd.4 349 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:28.934 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:39:28.930+0000 16949700 -1 osd.7 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:56.938592+0000 front 2019-11-25T16:37:57.144882+0000 (oldest deadline 2019-11-25T16:38:42.750759+0000) 2019-11-25T16:39:29.022 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:39:29.018+0000 16949700 -1 osd.0 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:58.443934+0000 front 2019-11-25T16:37:58.441737+0000 (oldest deadline 2019-11-25T16:38:43.333824+0000) 2019-11-25T16:39:29.153 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:39:29.149+0000 16949700 -1 osd.1 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.606203+0000 front 2019-11-25T16:38:01.137970+0000 (oldest deadline 2019-11-25T16:38:45.083811+0000) 2019-11-25T16:39:29.159 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:39:29.156+0000 16949700 -1 osd.5 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.764287+0000 front 2019-11-25T16:38:00.612639+0000 (oldest deadline 2019-11-25T16:38:44.081906+0000) 2019-11-25T16:39:29.173 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:39:29.168+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e349 v349) 2019-11-25T16:39:29.300 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:39:29.294+0000 16949700 -1 osd.3 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:38:00.750211+0000 front 2019-11-25T16:37:57.165812+0000 (oldest deadline 2019-11-25T16:38:40.657084+0000) 2019-11-25T16:39:29.623 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:39:29.619+0000 16949700 -1 osd.2 349 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:37:57.316235+0000 front 2019-11-25T16:37:57.305963+0000 (oldest deadline 2019-11-25T16:38:42.274105+0000) 2019-11-25T16:39:29.879 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:39:29.863+0000 16949700 -1 osd.4 350 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e348) 2019-11-25T16:39:34.381 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:24:38.964 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:34.417 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:24:39.023 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:34.418 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:24:39.024 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:34.433 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:24:39.039 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:34.433 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:24:39.039 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:34.448 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:24:39.031 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:34.543 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:24:39.149 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:34.545 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:24:39.128 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:34.549 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:24:39.155 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:39:58.824 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:39:58.801+0000 37a75700 -1 osd.6 354 build_incremental_map_msg missing incremental map 354 2019-11-25T16:39:59.047 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:39:59.023+0000 3aa7b700 -1 osd.6 354 build_incremental_map_msg missing incremental map 354 2019-11-25T16:39:59.232 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:39:59.108+0000 37274700 -1 osd.6 354 build_incremental_map_msg missing incremental map 354 2019-11-25T16:39:59.668 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:39:59.340+0000 3a27a700 -1 osd.6 354 build_incremental_map_msg missing incremental map 354 2019-11-25T16:39:59.671 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:39:59.422+0000 35a71700 -1 osd.6 354 build_incremental_map_msg missing incremental map 354 2019-11-25T16:39:59.680 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:39:59.486+0000 3aa7b700 -1 osd.6 354 build_incremental_map_msg missing incremental map 354 2019-11-25T16:40:09.721 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:40:09.714+0000 3aa7b700 -1 osd.1 355 build_incremental_map_msg missing incremental map 318 2019-11-25T16:40:40.869 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:40:40.515+0000 3426e700 -1 osd.4 355 build_incremental_map_msg missing incremental map 348 2019-11-25T16:40:45.611 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:40:45.607+0000 36272700 -1 osd.6 355 build_incremental_map_msg missing incremental map 329 2019-11-25T16:40:45.616 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:40:45.611+0000 36272700 -1 osd.6 355 build_incremental_map_msg missing incremental map 342 2019-11-25T16:40:45.621 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:40:45.616+0000 36272700 -1 osd.6 355 build_incremental_map_msg missing incremental map 354 2019-11-25T16:41:02.955 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:02.950+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:03.111 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:03.106+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:03.442 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:03.438+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:03.510 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:03.506+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:03.940 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:03.934+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:04.104 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:04.098+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:04.272 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:04.267+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:04.432 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:04.428+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:04.510 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:04.504+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:04.906 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:04.900+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:05.099 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:05.093+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:05.286 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:05.280+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:05.404 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:05.399+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:05.536 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:05.532+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:05.899 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:05.894+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:06.032 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:06.026+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:06.254 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:06.247+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:06.283 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:06.277+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:06.434 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:06.428+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:06.547 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:06.542+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:06.934 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:06.929+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:07.099 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:07.093+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:07.209 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:07.205+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:07.303 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:07.299+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:07.412 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:07.407+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:07.568 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:07.563+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:07.947 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:07.943+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:08.118 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:08.114+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:08.324 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:08.319+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:08.347 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:08.343+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:08.378 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:08.373+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:08.529 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:08.522+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:08.987 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:08.980+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:09.085 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:09.080+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:09.311 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:09.305+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:09.320 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:09.314+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:09.399 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:09.394+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:09.568 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:09.563+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:09.973 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:09.968+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:10.116 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:10.111+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:10.304 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:10.300+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:10.361 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:10.356+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:10.402 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:10.397+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:10.537 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:10.532+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:10.687 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:10.683+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:10.982 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:10.978+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:11.135 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:11.130+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:11.304 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:11.301+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:11.387 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:11.382+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:11.402 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:11.397+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:11.568 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:11.563+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:11.734 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:11.729+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:11.962 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:11.958+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:12.174 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:12.169+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:12.328 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:12.324+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:12.435 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:12.430+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:12.455 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:12.450+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:12.539 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:12.535+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:12.700 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:12.694+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:12.986 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:12.981+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:13.169 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:13.164+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:13.301 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:13.296+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:13.435 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:13.429+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:13.457 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:13.451+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:13.505 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:13.500+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:13.681 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:13.676+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:13.977 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:13.973+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:14.212 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:14.208+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:14.302 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:14.298+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:14.440 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:14.435+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:14.443 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:14.439+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:14.498 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:14.492+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:14.666 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:14.660+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:14.999 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:14.993+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:15.240 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:15.236+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:15.346 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:15.341+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:15.351 INFO:teuthology.orchestra.run.smithi153.stderr:testIfMatch (test.functional.tests.TestFileComparison) ... ok 2019-11-25T16:41:15.424 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:15.420+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:15.506 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:15.501+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:15.562 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:15.556+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:15.643 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:15.638+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:15.976 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:15.973+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:16.209 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:16.205+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:16.356 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:16.352+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:16.412 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:16.405+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:16.519 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:16.513+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:16.579 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:16.566+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:16.661 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:16.657+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:17.010 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:17.005+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:17.227 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:17.222+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:17.343 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:17.339+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:17.399 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:17.394+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:17.506 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:17.501+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:17.610 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:17.604+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:17.642 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:17.637+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:18.024 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:18.020+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:18.277 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:18.273+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:18.325 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:18.321+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:18.422 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:18.414+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:18.529 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:18.524+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:18.651 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:18.646+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:18.651 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:18.646+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:19.040 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:19.035+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:19.277 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:19.272+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:19.321 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:19.316+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:19.393 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:19.389+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:19.510 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:19.505+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:19.671 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:19.666+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:19.694 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:19.689+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:20.027 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:20.021+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:20.262 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:20.256+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:20.347 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:20.341+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:20.389 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:20.384+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:20.516 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:20.510+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:20.649 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:20.644+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:20.714 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:20.709+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:20.989 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:20.984+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:21.279 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:21.273+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:21.303 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:21.298+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:21.362 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:21.357+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:21.554 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:21.549+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:21.644 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:21.640+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:21.758 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:21.753+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:22.044 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:22.038+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:22.278 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:22.273+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:22.282 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:22.277+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:22.328 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:22.323+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:22.518 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:22.514+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:22.669 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:22.663+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:22.723 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:22.719+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:23.063 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:23.058+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:23.256 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:23.250+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:23.258 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:23.253+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:23.349 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:23.344+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:23.498 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:23.493+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:23.711 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:23.706+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:23.713 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:23.707+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:24.036 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:24.028+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:24.244 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:24.238+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:24.302 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:24.296+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:24.323 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:24.317+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:24.544 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:24.539+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:24.683 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:24.679+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:24.754 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:24.748+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:25.071 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:25.065+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:25.258 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:25.252+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:25.288 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:25.279+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:25.344 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:25.338+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:25.584 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:25.579+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:25.693 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:25.689+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:25.784 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:25.779+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:26.075 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:26.069+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:26.291 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:26.285+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:26.312 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:26.306+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:26.327 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:26.315+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:26.583 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:26.577+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:26.671 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:26.666+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:26.787 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:26.783+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:27.105 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:27.100+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:27.288 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:27.282+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:27.296 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:27.290+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:27.319 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:27.314+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:27.602 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:27.598+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:27.628 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:27.624+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:27.797 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:27.788+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:28.136 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:28.132+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:28.265 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:28.259+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:28.283 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:28.278+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:28.336 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:28.323+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:28.632 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:28.627+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:28.650 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:28.644+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:28.844 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:28.838+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:29.128 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:29.123+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:29.246 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:29.241+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:29.321 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:29.316+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:29.356 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:29.349+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:29.618 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:29.613+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:29.639 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:29.634+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:29.877 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:29.873+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:30.111 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:30.104+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:30.289 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:30.284+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:30.291 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:30.285+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:30.376 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:30.372+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:30.587 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:30.582+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:30.622 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:30.619+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:30.902 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:30.897+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:31.074 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:31.068+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:31.267 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:31.261+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:31.275 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:31.269+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:31.405 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:31.400+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:31.598 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:31.593+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:31.644 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:31.639+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:31.954 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:31.948+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:32.091 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:32.085+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:32.234 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:32.229+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:32.320 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:32.312+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:32.384 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:32.378+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:32.636 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:32.631+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:32.644 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:32.638+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:32.992 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:32.987+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:33.098 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:33.092+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:33.196 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:33.190+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:33.315 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:33.310+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:33.434 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:33.428+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:33.609 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:33.605+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:33.678 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:33.673+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:33.985 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:33.979+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:34.119 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:34.114+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:34.180 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:34.175+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:34.333 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:34.328+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:34.479 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:34.474+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:34.617 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:34.611+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:34.681 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:34.677+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:34.699 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:41:34.694+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 42sec e355 v355) 2019-11-25T16:41:34.995 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:34.990+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:35.100 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:35.094+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:35.144 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:35.137+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:35.383 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:35.378+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:35.480 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:35.473+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:35.583 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:35.578+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:35.696 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:35.690+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:36.002 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:35.994+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:36.075 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:36.070+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:36.170 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:36.164+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:36.379 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:36.374+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:36.499 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:36.494+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:36.620 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:36.613+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:36.718 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:36.712+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:37.033 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:37.027+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:37.061 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:37.055+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:37.215 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:37.209+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:37.339 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:37.335+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:37.510 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:37.504+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:37.672 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:37.666+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:37.700 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:37.695+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:38.018 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:38.012+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:38.097 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:38.092+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:38.214 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:38.208+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:38.371 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:38.366+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:38.534 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:38.529+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:38.667 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:38.662+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:38.717 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:38.711+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:39.024 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:39.019+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:39.105 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:39.100+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:39.221 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:39.215+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:39.398 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:39.393+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:39.576 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:39.571+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:39.692 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:39.688+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:39.696 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:39.691+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:39.711 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:41:39.707+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 42sec e355 v355) 2019-11-25T16:41:39.995 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:39.991+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:40.134 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:40.128+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:40.183 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:40.178+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:40.364 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:40.358+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:40.623 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:40.616+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:40.661 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:40.656+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:40.718 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:40.714+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:41.026 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:41.021+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:41.118 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:41.112+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:41.221 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:41.216+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:41.358 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:41.352+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:41.632 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:41.627+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:41.635 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:41.629+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:41.719 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:41.714+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:42.047 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:42.041+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:42.084 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:42.078+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:42.204 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:42.199+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:42.325 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:42.319+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:42.610 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:42.605+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:42.639 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:42.634+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:42.767 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:42.762+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:43.036 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:43.030+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:43.099 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:43.092+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:43.259 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:43.254+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:43.291 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:43.283+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:43.608 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:43.604+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:43.660 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:43.654+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:43.802 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:43.797+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:44.024 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:44.019+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:41:44.095 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:41:44.089+0000 16949700 -1 osd.5 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.449139+0000 front 2019-11-25T16:40:26.017455+0000 (oldest deadline 2019-11-25T16:41:01.992155+0000) 2019-11-25T16:41:44.238 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:41:44.232+0000 16949700 -1 osd.3 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:26.017639+0000 front 2019-11-25T16:40:19.493596+0000 (oldest deadline 2019-11-25T16:41:02.950706+0000) 2019-11-25T16:41:44.283 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:41:44.278+0000 16949700 -1 osd.7 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.441441+0000 front 2019-11-25T16:40:21.401853+0000 (oldest deadline 2019-11-25T16:41:05.474788+0000) 2019-11-25T16:41:44.611 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:41:44.606+0000 16949700 -1 osd.2 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.841076+0000 front 2019-11-25T16:40:21.190546+0000 (oldest deadline 2019-11-25T16:41:03.479509+0000) 2019-11-25T16:41:44.705 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:41:44.699+0000 16949700 -1 osd.6 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.844804+0000 front 2019-11-25T16:40:21.839800+0000 (oldest deadline 2019-11-25T16:41:02.441108+0000) 2019-11-25T16:41:44.784 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:41:44.775+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 42sec e355 v355) 2019-11-25T16:41:44.828 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:41:44.821+0000 16949700 -1 osd.0 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:21.396333+0000 front 2019-11-25T16:40:20.419520+0000 (oldest deadline 2019-11-25T16:41:03.568948+0000) 2019-11-25T16:41:44.996 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:41:44.992+0000 16949700 -1 osd.1 355 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:40:25.172080+0000 front 2019-11-25T16:40:25.640020+0000 (oldest deadline 2019-11-25T16:41:10.007670+0000) 2019-11-25T16:42:15.460 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:15.454+0000 16949700 -1 osd.3 357 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:118061 2.34s0 2:2d89bacd:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.656_4b56cd48eb0a4f2ba3759bcedf94626a:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e355) 2019-11-25T16:42:15.598 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:15.590+0000 16949700 -1 osd.7 357 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:118060 7.f 7:f315d535:::2019-11-25-16-103c5a94-eb69-4977-8c90-9f6d634d8761.4358.656-bdb696ac1b3d40ac88b1f474df26385a:head [append 0~525 in=525b] snapc 0=[] ondisk+write+known_if_redirected e355) 2019-11-25T16:42:21.580 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:27:26.163 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:21.588 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:27:26.171 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:21.590 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:27:26.196 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:21.593 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:27:26.199 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:21.595 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:27:26.201 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:21.599 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:27:26.205 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:21.761 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:27:26.367 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:21.776 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:27:26.359 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:21.802 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:27:26.408 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:42:24.223 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:42:24.204+0000 38276700 -1 osd.6 359 build_incremental_map_msg missing incremental map 359 2019-11-25T16:42:24.325 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:42:24.218+0000 36272700 -1 osd.6 359 build_incremental_map_msg missing incremental map 359 2019-11-25T16:42:24.712 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:42:24.622+0000 36272700 -1 osd.6 359 build_incremental_map_msg missing incremental map 359 2019-11-25T16:42:24.931 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:42:24.778+0000 3b27c700 -1 osd.6 359 build_incremental_map_msg missing incremental map 359 2019-11-25T16:42:40.919 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:40.913+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:41.908 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:41.902+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:42.144 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:42.139+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:42.264 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:42.260+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:42.865 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:42.860+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:43.115 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:43.109+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:43.315 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:43.310+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:43.869 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:43.861+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:43.889 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:43.883+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:44.079 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:44.074+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:44.301 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:44.296+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:44.851 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:44.845+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:44.922 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:44.916+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:44.923 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:44.917+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:45.062 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:45.057+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:45.272 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:45.266+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:45.827 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:45.822+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:45.935 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:45.929+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:45.944 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:45.938+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:46.036 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:46.031+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:46.275 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:46.269+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:46.808 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:46.803+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:46.902 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:46.895+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:46.937 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:46.932+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:47.043 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:47.038+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:47.269 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:47.262+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:47.769 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:47.764+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:47.858 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:47.852+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:47.959 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:47.953+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:48.076 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:48.072+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:48.285 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:48.280+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:48.794 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:48.790+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:48.860 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:48.854+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:48.991 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:48.985+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:49.066 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:49.062+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:49.271 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:49.266+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:49.768 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:49.764+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:49.870 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:49.864+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:49.957 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:49.952+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:50.044 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:50.040+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:50.323 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:50.318+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:50.808 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:50.804+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:50.865 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:50.860+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:50.938 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:50.934+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:51.016 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:51.012+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:51.314 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:51.309+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:51.848 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:51.843+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:51.862 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:51.858+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:51.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:51.961+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:52.014 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:52.009+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:52.293 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:52.287+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:52.831 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:52.826+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:52.897 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:52.891+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:52.997 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:52.993+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:53.051 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:53.046+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:53.321 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:53.316+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:53.878 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:53.873+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:53.879 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:53.875+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:53.997 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:53.993+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:54.031 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:54.027+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:54.324 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:54.319+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:54.397 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:42:54.394+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:42:54.851 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:54.847+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:54.908 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:54.903+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:54.994 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:54.989+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:55.027 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:55.021+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:55.312 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:55.308+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:55.358 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:42:55.353+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:42:55.878 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:55.873+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:55.879 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:55.874+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:55.987 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:55.983+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:56.001 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:55.996+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:56.277 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:56.273+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:56.400 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:42:56.397+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:42:56.879 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:56.874+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:56.895 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:56.890+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:56.997 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:56.993+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:57.014 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:57.009+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:57.274 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:57.270+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:57.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:42:57.381+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:42:57.886 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:57.881+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:57.887 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:57.883+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:57.991 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:57.987+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:58.034 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:58.029+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:58.254 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:58.249+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:58.413 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:42:58.410+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:42:58.884 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:58.879+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:58.911 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:58.906+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:42:58.988 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:42:58.984+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:42:59.052 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:42:59.047+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:42:59.260 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:42:59.254+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:42:59.404 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:42:59.400+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:42:59.838 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:42:59.834+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:42:59.963 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:42:59.958+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:00.013 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:00.009+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:00.022 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:00.018+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:00.223 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:00.218+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:00.454 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:00.450+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:00.835 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:00.829+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:00.964 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:00.959+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:01.019 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:01.013+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:01.032 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:01.027+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:01.192 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:01.185+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:01.472 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:01.465+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:01.473 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:01.467+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:01.726 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:01.721+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:01.869 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:01.863+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:01.988 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:01.983+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:01.994 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:01.989+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:02.074 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:02.069+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:02.146 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:02.140+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:02.512 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:02.505+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:02.512 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:02.507+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:02.755 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:02.750+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:02.872 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:02.865+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:02.963 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:02.957+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:02.971 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:02.966+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:03.071 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:03.066+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:03.141 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:03.136+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:03.493 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:03.487+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:03.493 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:03.489+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:03.738 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:03.734+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:03.920 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:03.914+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:03.980 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:03.975+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:04.024 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:04.019+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:04.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:04.077+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:04.103 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:04.095+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:04.548 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:04.542+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:04.556 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:04.547+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:04.719 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:04.711+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:04.966 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:04.961+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:05.012 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:05.007+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:05.048 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:05.043+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:05.069 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:05.064+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:05.153 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:05.149+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:05.520 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:05.513+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:05.521 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:05.515+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:05.751 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:05.745+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:05.951 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:05.946+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:06.018 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:06.013+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:06.029 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:06.024+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:06.105 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:06.101+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:06.201 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:06.196+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:06.566 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:06.559+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:06.567 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:06.561+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:06.706 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:06.702+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:06.921 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:06.915+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:06.996 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:06.990+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:07.065 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:07.060+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:07.077 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:07.072+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:07.181 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:07.176+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:07.551 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:07.545+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:07.552 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:07.547+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:07.745 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:07.740+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:07.906 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:07.899+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:07.953 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:07.949+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:08.044 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:08.040+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:08.130 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:08.126+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:08.170 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:08.166+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:08.529 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:08.523+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:08.535 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:08.529+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:08.713 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:08.707+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:08.926 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:08.920+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:08.951 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:08.946+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:09.071 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:09.066+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:09.123 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:09.118+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:09.205 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:09.199+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:09.535 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:09.529+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:09.536 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:09.529+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:09.688 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:09.684+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:09.966 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:09.961+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:09.974 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:09.968+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:10.033 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:10.029+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:10.106 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:10.101+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:10.179 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:10.175+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:10.532 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:10.526+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:10.532 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:10.527+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:10.679 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:10.674+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:10.921 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:10.915+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:10.940 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:10.933+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:11.073 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:11.068+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:11.119 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:11.110+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:11.188 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:11.185+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:11.578 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:11.572+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:11.579 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:11.574+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:11.688 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:11.684+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:11.909 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:11.903+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:11.942 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:11.936+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:12.078 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:12.070+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:12.122 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:12.118+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:12.192 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:12.188+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:12.593 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:12.586+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:12.596 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:12.588+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:12.733 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:12.729+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:12.903 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:12.898+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:12.949 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:12.943+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:13.051 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:13.046+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:13.154 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:13.150+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:13.198 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:13.194+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:13.565 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:13.560+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:13.566 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:13.561+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:13.747 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:13.741+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:13.917 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:13.911+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:13.972 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:13.966+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:14.047 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:14.042+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:14.175 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:14.170+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:14.205 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:14.201+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:14.607 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:14.601+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:14.608 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:14.602+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:14.768 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:14.763+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:14.954 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:14.948+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:15.002 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:14.997+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:15.065 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:43:15.057+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 39sec e361 v361) 2019-11-25T16:43:15.066 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:15.061+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:15.178 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:15.174+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:15.254 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:15.250+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:15.650 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:15.646+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:15.653 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:15.648+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:15.752 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:15.747+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:16.002 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:15.997+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:16.020 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:16.014+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:16.106 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:16.102+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:16.224 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:16.219+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:16.231 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:16.227+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:16.706 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:16.700+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:16.708 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:16.702+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:16.740 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:16.734+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:17.028 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:17.023+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:17.061 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:17.056+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:17.063 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:17.058+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:17.190 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:17.185+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:17.233 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:17.228+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:17.724 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:17.719+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:17.725 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:17.721+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:17.787 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:17.783+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:18.043 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:18.038+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:18.062 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:18.057+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:18.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:18.077+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:18.187 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:18.182+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:18.238 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:18.234+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:18.768 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:18.762+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:18.769 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:18.763+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:18.780 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:18.775+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:19.050 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:19.045+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:19.075 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:19.070+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:19.076 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:19.070+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:19.223 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:19.218+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:19.232 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:19.228+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:19.744 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:19.739+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:19.753 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:19.747+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:19.754 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:19.749+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:20.009 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:20.004+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:20.065 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:20.059+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:20.074 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:20.069+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:20.088 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:43:20.081+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 39sec e361 v361) 2019-11-25T16:43:20.203 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:20.198+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:20.218 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:20.214+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:20.746 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:20.741+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:20.746 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:20.741+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:20.750 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:20.745+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:20.981 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:20.976+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:21.056 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:21.051+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:21.095 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:21.089+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:21.239 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:21.233+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:21.246 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:21.241+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:21.713 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:21.709+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:21.716 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:21.711+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:21.717 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:21.712+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:21.938 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:21.933+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:22.049 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:22.044+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:22.092 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:22.087+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:22.210 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:22.206+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:22.266 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:22.261+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:22.701 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:22.695+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:22.702 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:22.697+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:22.739 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:22.734+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:22.910 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:22.905+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:23.049 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:23.044+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:23.106 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:23.101+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:23.256 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:23.252+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:23.299 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:23.294+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:23.669 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:23.664+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:23.671 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:23.667+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:23.786 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:23.781+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:23.911 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:23.906+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:24.038 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:24.033+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:24.099 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:24.093+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:24.223 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:24.218+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:24.325 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:24.321+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:24.655 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:24.649+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:24.656 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:24.651+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:24.791 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:24.786+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:24.946 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:24.941+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:25.037 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:25.032+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:25.109 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:43:25.101+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 39sec e361 v361) 2019-11-25T16:43:25.135 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:25.130+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:25.292 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:25.286+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:25.329 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:25.325+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:25.683 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:25.677+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:25.684 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:25.678+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:25.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:25.761+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:25.952 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:25.946+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:26.020 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:26.014+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:26.110 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:43:26.105+0000 16949700 -1 osd.5 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:02.324519+0000 front 2019-11-25T16:42:01.143849+0000 (oldest deadline 2019-11-25T16:42:40.853272+0000) 2019-11-25T16:43:26.275 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:26.270+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:26.299 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:26.294+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:26.638 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:26.632+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:26.639 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:26.633+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:26.753 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:26.744+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:26.939 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:26.934+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:26.975 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:26.970+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:27.277 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:27.272+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:27.351 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:27.346+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:27.623 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:27.617+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:27.623 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:27.618+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:27.785 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:27.780+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:27.955 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:27.950+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:28.012 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:28.006+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:28.291 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:28.287+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:28.355 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:28.350+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:28.643 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:28.637+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:28.644 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:28.639+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:28.827 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:28.823+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:28.946 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:28.941+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:29.046 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:29.041+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:29.253 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:29.248+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:29.310 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:29.306+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:29.622 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:29.618+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:29.624 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:29.620+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:29.861 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:29.857+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:29.904 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:29.899+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:30.011 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:30.006+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:30.120 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:43:30.116+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 39sec e361 v361) 2019-11-25T16:43:30.251 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:30.246+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:30.326 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:30.318+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:30.596 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:30.590+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:30.596 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:30.591+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:30.840 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:30.835+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:30.883 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:30.878+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:31.043 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:31.039+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:31.267 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:31.261+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:31.346 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:31.341+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:31.583 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:31.577+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:31.584 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:31.578+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:31.844 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:31.840+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:31.902 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:31.897+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:32.057 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:32.053+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:32.299 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:32.293+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:32.389 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:32.384+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:32.590 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:32.583+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:32.592 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:32.585+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:32.851 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:32.846+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:32.956 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:32.950+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:33.099 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:33.094+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:33.298 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:33.293+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:33.414 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:33.408+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:33.601 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:33.596+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:33.601 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:33.597+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:33.884 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:33.879+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:33.994 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:33.988+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:34.059 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:34.054+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:34.273 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:34.269+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:34.409 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:34.403+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:34.583 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:34.579+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:34.587 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:34.582+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:34.885 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:34.880+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:35.029 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:35.023+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:35.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:35.077+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:35.139 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:43:35.133+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 39sec e361 v361) 2019-11-25T16:43:35.327 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:35.322+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:35.384 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:35.374+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:35.631 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:35.622+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:35.631 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:35.624+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:35.903 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:35.897+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:36.010 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:36.005+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:36.077 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:36.070+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:36.337 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:36.331+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:36.437 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:36.432+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:36.625 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:36.620+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:36.628 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:36.624+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:36.932 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:36.926+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:37.051 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:37.045+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:37.054 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:37.048+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:37.353 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:37.349+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:37.438 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:37.432+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:37.674 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:37.667+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:37.674 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:37.669+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:37.921 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:37.915+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:38.072 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:38.065+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:38.082 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:38.077+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:38.398 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:38.392+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:38.448 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:38.442+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:38.670 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:38.666+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:38.673 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:38.669+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:38.956 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:38.950+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:39.084 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:39.078+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:39.086 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:39.082+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:39.360 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:39.354+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:39.441 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:43:39.428+0000 16949700 -1 osd.0 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:57.561260+0000 front 2019-11-25T16:41:57.535909+0000 (oldest deadline 2019-11-25T16:42:42.203391+0000) 2019-11-25T16:43:39.649 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:39.645+0000 16949700 -1 osd.4 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:42:21.254325+0000 (oldest deadline 2019-11-25T16:43:01.254325+0000) 2019-11-25T16:43:39.652 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:43:39.649+0000 16949700 -1 osd.4 361 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e359) 2019-11-25T16:43:39.982 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:43:39.976+0000 16949700 -1 osd.1 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:42:00.943531+0000 front 2019-11-25T16:42:02.322749+0000 (oldest deadline 2019-11-25T16:43:01.264724+0000) 2019-11-25T16:43:40.055 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:43:40.049+0000 16949700 -1 osd.3 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.257158+0000 front 2019-11-25T16:41:59.256002+0000 (oldest deadline 2019-11-25T16:42:43.335930+0000) 2019-11-25T16:43:40.122 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:43:40.117+0000 16949700 -1 osd.7 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.282224+0000 front 2019-11-25T16:41:59.277067+0000 (oldest deadline 2019-11-25T16:42:42.118592+0000) 2019-11-25T16:43:40.225 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:43:40.210+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 39sec e361 v361) 2019-11-25T16:43:40.375 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:43:40.371+0000 16949700 -1 osd.2 361 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:41:59.153079+0000 front 2019-11-25T16:41:59.169351+0000 (oldest deadline 2019-11-25T16:42:44.029056+0000) 2019-11-25T16:43:56.910 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:43:56.906+0000 34a6f700 -1 osd.6 361 build_incremental_map_msg missing incremental map 359 2019-11-25T16:44:03.462 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:44:03.453+0000 37274700 -1 osd.6 366 build_incremental_map_msg missing incremental map 366 2019-11-25T16:44:03.480 INFO:teuthology.orchestra.run.smithi153.stderr:testIfMatchAndUnmodified (test.functional.tests.TestFileComparison) ... ok 2019-11-25T16:44:31.503 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:44:31.499+0000 36272700 -1 osd.6 366 build_incremental_map_msg missing incremental map 359 2019-11-25T16:44:31.508 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:44:31.503+0000 36272700 -1 osd.6 366 build_incremental_map_msg missing incremental map 366 2019-11-25T16:45:08.006 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:08.001+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:08.565 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:08.560+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:08.842 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:08.837+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:08.887 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:08.882+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:09.012 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:09.006+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:09.591 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:09.585+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:09.745 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:09.741+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:09.859 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:09.853+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:09.871 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:09.865+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:09.997 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:09.992+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:10.547 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:10.542+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:10.703 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:10.697+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:10.843 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:10.838+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:10.879 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:10.872+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:10.980 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:10.976+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:11.562 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:11.555+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:11.670 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:11.664+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:11.850 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:11.844+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:11.894 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:11.888+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:11.950 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:11.945+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:12.541 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:12.535+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:12.644 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:12.638+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:12.894 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:12.887+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:12.928 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:12.917+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:12.940 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:12.936+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:13.560 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:13.554+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:13.622 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:13.617+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:13.918 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:13.913+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:13.919 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:13.914+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:13.978 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:13.973+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:14.536 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:14.531+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:14.617 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:14.611+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:14.903 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:14.897+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:14.993 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:14.986+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:15.002 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:14.978+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:15.429 INFO:teuthology.orchestra.run.smithi153.stderr:testIfModifiedSince (test.functional.tests.TestFileComparison) ... ok 2019-11-25T16:45:15.521 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:15.516+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:15.571 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:15.567+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:15.874 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:15.864+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:15.955 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:15.950+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:15.990 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:15.986+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:16.490 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:16.484+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:16.613 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:16.608+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:16.909 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:16.903+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:16.939 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:16.932+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:16.995 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:16.990+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:17.555 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:17.464+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:17.572 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:17.567+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:17.882 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:17.877+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:17.937 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:17.932+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:17.987 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:17.983+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:18.512 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:18.506+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:18.616 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:18.611+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:18.870 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:18.864+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:18.905 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:18.899+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:19.010 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:19.004+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:19.498 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:19.494+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:19.578 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:19.573+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:19.908 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:19.903+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:19.936 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:19.932+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:20.032 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:20.028+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:20.555 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:20.549+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:20.615 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:20.611+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:20.916 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:20.912+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:20.936 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:20.931+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:21.023 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:21.019+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:21.556 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:21.552+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:21.640 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:21.636+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:21.895 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:21.889+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:21.911 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:21.906+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:22.028 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:22.021+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:22.584 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:22.580+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:22.665 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:22.661+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:22.932 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:22.927+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:22.938 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:22.932+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:23.047 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:23.041+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:23.578 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:23.573+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:23.683 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:23.675+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:23.914 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:23.909+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:23.962 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:23.955+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:24.044 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:24.037+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:24.119 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:24.113+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:24.535 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:24.530+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:24.668 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:24.663+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:24.916 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:24.911+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:24.969 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:24.963+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:25.043 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:25.038+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:25.094 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:25.088+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:25.552 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:25.547+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:25.654 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:25.650+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:25.883 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:25.877+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:26.004 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:25.998+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:26.063 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:26.056+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:26.088 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:26.082+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:26.529 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:26.524+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:26.629 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:26.624+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:26.878 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:26.873+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:27.020 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:27.013+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:27.103 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:27.097+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:27.123 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:27.118+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:27.525 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:27.520+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:27.674 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:27.669+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:27.866 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:27.862+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:28.008 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:28.002+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:28.103 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:28.097+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:28.172 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:28.168+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:28.522 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:28.517+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:28.701 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:28.696+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:28.916 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:28.910+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:29.018 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:29.013+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:29.124 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:29.119+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:29.155 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:29.149+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:29.576 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:29.568+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:29.694 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:29.690+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:29.912 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:29.907+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:30.012 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:30.007+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:30.122 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:30.117+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:30.141 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:30.136+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:30.615 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:30.609+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:30.692 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:30.687+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:30.884 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:30.879+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:31.058 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:31.053+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:31.087 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:31.082+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:31.192 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:31.187+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:31.598 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:31.593+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:31.705 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:31.701+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:31.842 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:31.837+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:32.058 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:32.053+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:32.132 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:32.127+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:32.223 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:32.217+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:32.586 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:32.580+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:32.748 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:32.743+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:32.853 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:32.846+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:33.105 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:33.101+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:33.145 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:33.139+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:33.178 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:33.172+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:33.610 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:33.604+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:33.800 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:33.794+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:33.865 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:33.858+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:34.088 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:34.083+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:34.184 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:34.179+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:34.194 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:34.189+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:34.648 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:34.642+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:34.777 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:34.772+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:34.850 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:34.845+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:35.112 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:35.107+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:35.113 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:35.107+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:35.172 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:35.167+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:35.231 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:35.226+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:35.612 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:35.607+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:35.742 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:35.736+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:35.876 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:35.871+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:36.082 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:36.076+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:36.110 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:36.105+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:36.136 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:36.130+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:36.228 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:36.222+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:36.607 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:36.600+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:36.752 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:36.746+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:36.890 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:36.885+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:37.066 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:37.061+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:37.087 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:37.082+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:37.145 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:37.140+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:37.238 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:37.232+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:37.617 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:37.612+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:37.788 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:37.783+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:37.921 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:37.915+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:38.045 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:38.039+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:38.060 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:38.055+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:38.152 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:38.147+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:38.201 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:38.196+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:38.633 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:38.624+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:38.834 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:38.829+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:38.959 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:38.955+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:39.035 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:39.031+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:39.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:39.075+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:39.174 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:39.169+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:39.244 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:39.239+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:39.645 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:39.639+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:39.868 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:39.863+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:39.926 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:39.921+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:40.012 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:40.006+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:40.047 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:40.042+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:40.187 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:40.182+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:40.211 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:40.207+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:40.620 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:45:40.615+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 40sec e366 v366) 2019-11-25T16:45:40.655 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:40.650+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:40.879 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:40.874+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:40.890 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:40.884+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:41.012 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:41.007+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:41.021 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:41.016+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:41.188 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:41.183+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:41.218 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:41.213+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:41.658 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:41.653+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:41.863 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:41.856+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:41.933 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:41.926+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:42.011 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:42.007+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:42.070 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:41.998+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:42.174 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:42.168+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:42.209 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:42.204+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:42.688 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:42.682+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:42.900 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:42.895+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:42.939 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:42.917+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:43.048 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:43.043+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:43.115 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:43.110+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:43.218 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:43.213+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:43.227 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:43.222+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:43.727 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:43.720+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:43.863 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:43.858+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:43.990 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:43.983+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:44.007 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:44.001+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:44.113 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:44.108+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:44.231 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:44.226+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:44.232 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:44.227+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:44.779 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:44.773+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:45:44.919 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:45:44.914+0000 16949700 -1 osd.6 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.940135+0000 front 2019-11-25T16:44:27.636848+0000 (oldest deadline 2019-11-25T16:45:09.536771+0000) 2019-11-25T16:45:44.973 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:45:44.967+0000 16949700 -1 osd.7 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:23.657528+0000 front 2019-11-25T16:44:23.659181+0000 (oldest deadline 2019-11-25T16:45:07.932051+0000) 2019-11-25T16:45:45.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:45:45.025+0000 16949700 -1 osd.0 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:22.754588+0000 front 2019-11-25T16:44:26.345592+0000 (oldest deadline 2019-11-25T16:45:08.410725+0000) 2019-11-25T16:45:45.152 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:45:45.147+0000 16949700 -1 osd.3 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.907254+0000 front 2019-11-25T16:44:27.704683+0000 (oldest deadline 2019-11-25T16:45:34.099772+0000) 2019-11-25T16:45:45.188 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:45:45.182+0000 16949700 -1 osd.2 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:26.406838+0000 front 2019-11-25T16:44:27.432155+0000 (oldest deadline 2019-11-25T16:45:07.131452+0000) 2019-11-25T16:45:45.213 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:45:45.208+0000 16949700 -1 osd.1 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.705168+0000 front 2019-11-25T16:44:26.347809+0000 (oldest deadline 2019-11-25T16:45:24.106328+0000) 2019-11-25T16:45:45.704 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:45:45.691+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 40sec e366 v366) 2019-11-25T16:45:45.757 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:45:45.752+0000 16949700 -1 osd.5 366 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:44:27.991231+0000 front 2019-11-25T16:44:27.593930+0000 (oldest deadline 2019-11-25T16:45:07.573839+0000) 2019-11-25T16:46:20.989 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:31:25.571 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:21.000 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:31:25.583 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:21.019 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:31:25.625 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:21.024 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:31:25.630 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:21.042 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:31:25.648 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:21.069 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:31:25.675 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:21.242 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:31:25.825 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:21.299 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:31:25.905 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:21.324 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:31:25.930 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:23.591 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:46:23.556+0000 16949700 -1 osd.4 368 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:118652 2.es0 2:71d1ec35:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.656_8900c68759f144818acec07bf6421c83:head [getxattrs out=1002b,stat out=16b,read 0~132] snapc 0=[] ondisk+read+known_if_redirected e366) 2019-11-25T16:46:25.717 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:25.666+0000 37274700 -1 osd.1 370 build_incremental_map_msg missing incremental map 370 2019-11-25T16:46:25.798 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:25.792+0000 35270700 -1 osd.1 370 build_incremental_map_msg missing incremental map 370 2019-11-25T16:46:26.085 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:31:30.668 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:26.088 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:31:30.670 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:26.153 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:31:30.758 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:26.157 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:31:30.740 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:26.169 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:31:30.767 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:26.179 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:31:30.785 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:26.193 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:31:30.799 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:26.199 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:31:30.805 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:26.224 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:31:30.830 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:46:40.386 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:40.380+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:41.344 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:41.338+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:41.966 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:41.961+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:42.382 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:42.378+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:43.009 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:43.004+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:43.358 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:43.352+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:43.824 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:43.819+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:44.017 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:44.011+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:44.376 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:44.368+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:44.731 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:44.726+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:44.842 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:44.838+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:44.975 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:44.970+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:45.103 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:45.097+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:45.331 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:45.326+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:45.621 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:45.616+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:45.722 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:45.715+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:45.847 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:45.842+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:46.021 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:46.016+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:46.057 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:46.052+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:46.360 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:46.355+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:46.614 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:46.610+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:46.697 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:46.692+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:46.843 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:46.838+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:46.978 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:46.973+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:47.028 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:47.022+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:47.406 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:47.400+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:47.643 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:47.638+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:47.709 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:47.704+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:47.871 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:47.866+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:47.994 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:47.988+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:48.034 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:48.029+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:48.451 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:48.445+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:48.684 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:48.679+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:48.722 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:48.717+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:48.920 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:48.915+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:49.037 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:49.033+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:49.083 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:49.078+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:49.429 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:49.424+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:49.695 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:49.691+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:49.736 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:49.731+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:49.922 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:49.918+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:50.017 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:50.012+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:50.048 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:50.043+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:50.424 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:50.418+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:50.741 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:50.736+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:50.767 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:50.760+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:50.947 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:50.942+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:51.008 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:51.002+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:51.018 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:51.013+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:51.387 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:51.381+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:51.784 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:51.780+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:51.787 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:51.782+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:51.910 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:51.905+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:52.021 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:52.016+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:52.053 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:52.047+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:52.391 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:52.386+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:52.763 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:52.758+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:52.770 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:52.766+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:52.884 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:52.879+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:52.991 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:52.985+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:53.081 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:53.076+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:53.414 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:53.408+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:53.739 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:53.735+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:53.744 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:53.737+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:53.845 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:53.840+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:53.980 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:53.974+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:54.093 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:54.088+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:54.400 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:54.394+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:54.697 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:54.693+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:54.725 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:54.722+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:54.886 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:54.880+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:54.971 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:54.964+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:55.080 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:55.075+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:55.394 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:55.388+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:55.728 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:55.723+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:55.778 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:55.774+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:55.884 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:55.879+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:55.984 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:55.978+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:56.109 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:56.104+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:56.200 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:46:56.195+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:46:56.423 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:56.416+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:56.710 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:56.705+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:56.752 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:56.747+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:56.845 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:56.840+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:57.033 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:57.027+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:57.094 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:57.088+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:57.219 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:46:57.214+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:46:57.380 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:57.374+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:57.674 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:57.669+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:57.749 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:57.745+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:57.881 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:57.875+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:57.998 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:57.992+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:58.106 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:58.102+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:58.236 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:46:58.231+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:46:58.362 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:58.357+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:58.712 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:58.708+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:58.771 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:58.767+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:58.857 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:58.851+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:58.960 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:58.955+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:46:59.130 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:46:59.125+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:46:59.274 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:46:59.269+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:46:59.385 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:46:59.380+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:46:59.705 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:46:59.700+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:46:59.762 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:46:59.758+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:46:59.889 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:46:59.884+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:46:59.965 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:46:59.954+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:00.100 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:00.095+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:00.257 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:00.252+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:00.404 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:00.399+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:00.703 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:00.698+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:00.804 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:00.799+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:00.890 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:00.885+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:01.008 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:01.003+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:01.086 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:01.081+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:01.269 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:01.265+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:01.448 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:01.444+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:01.736 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:01.731+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:01.833 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:01.828+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:01.843 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:01.839+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:01.974 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:01.970+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:02.090 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:02.085+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:02.289 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:02.286+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:02.488 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:02.484+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:02.771 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:02.765+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:02.879 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:02.874+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:02.888 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:02.883+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:03.026 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:03.022+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:03.075 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:03.071+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:03.259 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:03.255+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:03.261 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:03.258+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:03.488 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:03.483+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:03.804 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:03.798+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:03.868 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:03.863+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:03.895 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:03.890+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:03.991 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:03.986+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:04.041 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:04.036+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:04.244 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:04.239+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:04.245 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:04.240+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:04.472 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:04.468+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:04.828 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:04.822+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:04.862 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:04.857+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:04.918 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:04.913+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:04.946 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:04.942+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:05.065 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:05.060+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:05.245 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:05.241+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:05.248 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:05.244+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:05.481 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:05.477+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:05.831 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:05.826+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:05.869 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:05.864+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:05.931 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:05.926+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:05.946 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:05.941+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:06.086 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:06.080+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:06.281 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:06.276+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:06.284 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:06.280+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:06.441 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:06.436+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:06.830 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:06.826+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:06.857 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:06.854+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:06.977 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:06.972+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:06.996 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:06.991+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:07.060 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:07.055+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:07.261 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:07.257+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:07.262 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:07.258+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:07.409 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:07.404+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:07.871 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:07.865+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:07.878 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:07.873+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:07.976 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:07.971+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:07.989 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:07.985+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:08.028 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:08.020+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:08.294 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:08.289+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:08.295 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:08.290+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:08.421 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:08.417+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:08.844 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:08.837+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:08.858 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:08.853+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:08.976 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:08.970+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:08.996 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:08.990+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:09.026 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:09.021+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:09.347 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:09.342+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:09.350 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:09.347+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:09.400 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:09.395+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:09.856 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:09.850+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:09.864 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:09.859+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:09.946 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:09.940+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:09.962 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:09.957+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:10.037 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:10.033+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:10.315 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:10.311+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:10.318 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:10.314+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:10.418 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:10.414+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:10.890 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:10.884+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:10.896 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:10.890+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:10.927 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:10.921+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:10.965 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:10.959+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:11.008 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:11.003+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:11.331 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:11.327+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:11.334 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:11.330+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:11.440 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:11.436+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:11.863 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:11.857+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:11.877 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:11.872+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:11.936 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:11.930+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:11.955 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:11.949+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:11.975 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:11.971+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:12.376 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:12.371+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:12.377 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:12.373+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:12.441 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:12.438+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:12.837 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:12.832+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:12.902 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:12.897+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:12.927 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:12.922+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:12.937 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:12.931+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:12.938 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:12.933+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:13.334 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:13.329+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:13.337 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:13.333+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:13.398 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:13.394+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:13.837 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:13.830+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:13.882 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:13.877+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:13.909 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:13.904+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:13.953 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:13.947+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:13.988 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:13.981+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:14.305 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:14.299+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:14.306 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:14.301+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:14.431 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:14.426+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:14.860 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:14.856+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:14.871 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:14.868+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:14.909 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:14.906+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:14.950 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:14.945+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:14.958 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:14.953+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:15.307 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:15.301+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:15.307 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:15.302+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:15.473 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:15.468+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:15.835 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:15.830+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:15.912 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:15.907+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:15.959 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:15.954+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:15.980 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:15.975+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:16.001 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:15.995+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:16.246 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:47:16.237+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e372 v372) 2019-11-25T16:47:16.333 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:16.328+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:16.334 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:16.329+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:16.471 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:16.464+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:16.794 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:16.789+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:16.908 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:16.905+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:16.971 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:16.967+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:16.974 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:16.970+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:17.023 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:17.018+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:17.380 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:17.374+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:17.381 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:17.376+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:17.462 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:17.456+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:17.797 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:17.792+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:17.958 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:17.953+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:17.996 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:17.991+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:18.021 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:18.014+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:18.032 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:18.028+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:18.343 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:18.337+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:18.345 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:18.339+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:18.484 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:18.479+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:18.760 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:18.755+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:18.959 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:18.955+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:18.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:18.960+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:19.056 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:19.049+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:19.065 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:19.059+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:19.353 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:19.348+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:19.355 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:19.351+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:19.464 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:19.459+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:19.741 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:19.737+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:19.980 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:19.976+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:20.009 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:20.005+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:20.108 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:20.103+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:20.109 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:20.104+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:20.359 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:20.353+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:20.360 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:20.355+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:20.513 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:20.507+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:20.749 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:20.743+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:20.986 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:20.980+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:20.998 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:20.994+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:21.097 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:21.092+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:21.143 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:21.138+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:21.260 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:47:21.252+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e372 v372) 2019-11-25T16:47:21.389 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:21.384+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:21.391 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:21.387+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:21.557 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:21.552+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:21.744 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:21.739+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:21.949 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:21.944+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:21.957 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:21.953+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:22.062 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:22.057+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:22.099 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:22.093+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:22.445 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:22.439+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:22.446 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:22.441+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:22.610 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:22.604+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:22.718 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:22.713+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:22.950 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:22.946+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:22.967 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:22.962+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:23.031 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:23.025+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:23.121 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:23.115+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:23.490 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:23.484+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:23.490 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:23.485+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:23.660 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:23.655+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:23.781 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:23.766+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:23.935 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:23.930+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:23.938 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:23.933+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:24.006 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:24.001+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:24.167 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:24.162+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:24.489 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:24.484+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:24.493 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:24.488+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:24.645 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:24.640+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:24.804 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:24.799+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:24.915 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:24.910+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:24.962 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:24.957+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:24.979 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:24.974+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:25.148 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:25.142+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:25.528 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:25.523+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:25.532 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:25.527+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:25.610 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:25.605+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:25.848 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:25.843+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:25.918 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:25.914+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:25.935 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:25.931+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:26.016 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:26.012+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:26.136 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:26.129+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:26.281 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:47:26.272+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e372 v372) 2019-11-25T16:47:26.541 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:26.536+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:26.546 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:26.539+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:26.622 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:26.616+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:26.817 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:26.813+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:26.954 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:26.950+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:26.968 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:26.963+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:27.034 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:27.029+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:27.158 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:27.152+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:27.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:27.558+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:27.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:27.559+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:27.597 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:27.592+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:27.840 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:27.835+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:27.959 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:27.954+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:27.998 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:27.993+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:28.003 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:27.998+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:28.193 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:28.187+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:28.554 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:28.548+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:28.555 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:28.549+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:28.618 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:28.613+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:28.814 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:28.810+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:28.914 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:28.909+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:29.026 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:29.022+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:29.039 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:29.035+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:29.152 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:29.147+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:29.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:29.557+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:29.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:29.559+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:29.580 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:29.574+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:29.811 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:29.806+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:29.900 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:29.895+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:30.042 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:30.033+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:30.044 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:30.038+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:30.129 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:30.123+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:30.571 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:30.567+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:30.598 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:30.593+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:30.599 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:30.594+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:30.771 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:30.766+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:30.869 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:30.865+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:31.002 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:30.997+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:31.081 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:31.077+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:31.165 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:31.160+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:31.292 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:47:31.288+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e372 v372) 2019-11-25T16:47:31.621 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:31.615+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:31.622 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:31.617+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:31.626 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:31.621+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:31.791 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:31.786+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:31.896 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:31.892+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:31.995 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:31.990+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:32.107 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:32.103+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:32.220 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:32.213+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:32.617 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:32.611+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:32.676 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:32.670+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:32.676 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:32.671+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:32.766 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:32.750+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:32.874 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:32.869+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:32.990 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:32.985+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:33.071 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:33.066+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:33.198 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:33.193+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:33.599 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:33.593+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:33.681 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:33.675+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:33.682 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:33.677+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:33.753 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:33.749+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:33.898 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:33.893+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:34.022 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:34.017+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:34.090 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:34.084+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:34.230 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:34.224+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:34.589 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:34.582+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:34.662 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:34.656+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:34.663 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:34.657+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:34.746 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:34.741+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:34.870 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:34.866+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:35.053 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:35.049+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:35.065 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:35.060+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:35.260 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:35.254+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:35.609 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:47:35.603+0000 16949700 -1 osd.7 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127126+0000 front 2019-11-25T16:45:59.581799+0000 (oldest deadline 2019-11-25T16:46:40.132410+0000) 2019-11-25T16:47:35.681 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:35.676+0000 16949700 -1 osd.4 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T16:46:23.222236+0000 (oldest deadline 2019-11-25T16:47:03.222236+0000) 2019-11-25T16:47:35.683 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:47:35.678+0000 16949700 -1 osd.4 372 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e370) 2019-11-25T16:47:35.771 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:35.766+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:35.918 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:35.914+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:36.028 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:47:36.023+0000 16949700 -1 osd.3 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:03.127047+0000 front 2019-11-25T16:46:00.406240+0000 (oldest deadline 2019-11-25T16:46:43.013649+0000) 2019-11-25T16:47:36.056 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:47:36.051+0000 16949700 -1 osd.0 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:00.199159+0000 front 2019-11-25T16:46:00.201132+0000 (oldest deadline 2019-11-25T16:46:45.574722+0000) 2019-11-25T16:47:36.271 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:47:36.266+0000 16949700 -1 osd.5 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:58.855518+0000 front 2019-11-25T16:45:59.070867+0000 (oldest deadline 2019-11-25T16:46:44.117019+0000) 2019-11-25T16:47:36.365 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:47:36.354+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e372 v372) 2019-11-25T16:47:36.770 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:47:36.765+0000 16949700 -1 osd.1 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:45:59.014403+0000 front 2019-11-25T16:46:03.071158+0000 (oldest deadline 2019-11-25T16:46:41.739248+0000) 2019-11-25T16:47:36.951 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:47:36.943+0000 16949700 -1 osd.2 372 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:46:01.319425+0000 front 2019-11-25T16:46:01.432201+0000 (oldest deadline 2019-11-25T16:46:44.426123+0000) 2019-11-25T16:47:56.546 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:33:01.128 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:56.581 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:33:01.187 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:56.587 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:33:01.170 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:56.600 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:33:01.206 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:56.628 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:33:01.233 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:56.644 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:33:01.250 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:56.740 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:33:01.346 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:56.748 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:33:01.330 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:56.835 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:33:01.441 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:47:58.572 INFO:teuthology.orchestra.run.smithi153.stderr:testIfNoneMatch (test.functional.tests.TestFileComparison) ... ok 2019-11-25T16:48:31.276 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:48:31.271+0000 36a73700 -1 osd.1 377 build_incremental_map_msg missing incremental map 370 2019-11-25T16:48:56.977 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:48:56.970+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:48:57.062 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:48:57.056+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:48:57.604 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:48:57.598+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:48:57.967 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:48:57.962+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:48:58.021 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:48:58.015+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:48:58.048 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:48:58.041+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:48:58.123 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:48:58.117+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:48:58.639 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:48:58.634+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:48:58.927 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:48:58.921+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:48:59.016 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:48:59.010+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:48:59.018 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:48:59.012+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:48:59.138 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:48:59.132+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:48:59.670 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:48:59.664+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:48:59.981 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:48:59.974+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:48:59.982 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:48:59.975+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:00.043 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:00.036+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:00.125 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:00.120+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:00.683 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:00.677+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:00.699 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:00.694+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:00.965 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:00.960+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:01.003 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:00.998+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:01.019 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:01.013+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:01.118 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:01.112+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:01.677 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:01.671+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:01.721 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:01.714+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:01.986 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:01.980+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:02.000 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:01.995+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:02.017 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:02.012+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:02.159 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:02.155+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:02.701 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:02.696+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:02.746 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:02.741+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:02.978 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:02.973+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:02.989 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:02.984+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:03.003 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:02.998+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:03.156 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:03.151+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:03.700 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:03.695+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:03.728 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:03.722+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:03.954 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:03.949+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:03.968 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:03.963+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:03.991 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:03.986+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:04.178 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:04.173+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:04.749 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:04.743+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:04.777 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:04.771+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:04.923 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:04.917+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:04.970 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:04.966+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:05.033 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:05.028+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:05.219 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:05.214+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:05.807 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:05.802+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:05.824 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:05.816+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:05.951 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:05.946+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:05.970 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:05.965+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:05.997 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:05.992+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:06.213 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:06.209+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:06.804 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:06.797+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:06.845 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:06.841+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:06.924 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:06.919+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:06.939 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:06.934+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:07.010 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:07.005+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:07.220 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:07.214+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:07.784 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:07.779+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:07.890 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:07.884+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:07.911 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:07.907+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:07.965 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:07.960+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:07.975 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:07.960+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:08.219 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:08.214+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:08.784 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:08.779+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:08.886 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:08.880+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:08.913 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:08.908+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:08.930 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:08.926+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:08.959 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:08.953+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:09.200 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:09.195+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:09.768 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:09.762+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:09.897 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:09.892+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:09.921 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:09.915+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:09.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:09.940+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:09.986 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:09.979+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:10.239 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:10.234+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:10.751 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:10.746+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:10.908 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:10.903+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:10.911 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:10.906+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:10.929 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:10.924+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:10.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:10.973+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:11.248 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:11.243+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:11.740 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:11.734+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:11.872 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:11.868+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:11.928 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:11.923+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:11.964 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:11.905+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:11.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:11.973+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:12.265 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:12.260+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:12.775 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:12.769+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:12.879 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:12.874+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:12.909 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:12.904+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:12.918 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:12.913+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:12.969 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:12.963+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:13.293 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:13.287+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:13.740 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:13.736+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:13.931 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:13.926+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:13.955 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:13.950+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:13.959 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:13.953+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:14.019 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:14.014+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:14.097 INFO:teuthology.orchestra.run.smithi153.stderr:testIfUnmodifiedSince (test.functional.tests.TestFileComparison) ... ok 2019-11-25T16:49:14.272 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:14.266+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:14.768 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:14.762+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:14.903 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:14.898+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:14.916 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:14.909+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:15.005 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:14.999+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:15.018 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:15.012+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:15.266 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:15.261+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:15.760 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:15.755+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:15.881 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:15.877+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:15.975 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:15.969+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:15.977 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:15.971+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:16.144 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:16.138+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:16.222 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:16.217+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:16.722 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:16.715+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:16.915 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:16.910+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:16.977 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:16.970+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:16.988 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:16.982+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:17.158 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:17.153+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:17.240 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:17.234+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:17.754 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:17.749+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:17.909 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:17.904+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:17.939 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:17.933+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:18.018 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:18.013+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:18.158 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:18.154+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:18.242 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:18.232+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:18.778 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:18.774+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:18.908 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:18.903+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:18.936 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:18.931+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:18.972 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:18.966+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:19.201 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:19.195+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:19.238 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:19.234+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:19.799 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:19.794+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:19.903 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:19.898+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:19.923 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:19.917+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:19.981 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:19.975+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:20.204 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:20.198+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:20.208 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:20.202+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:20.826 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:20.822+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:20.880 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:20.875+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:20.911 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:20.905+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:21.021 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:21.015+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:21.185 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:21.179+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:21.258 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:21.252+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:21.826 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:21.821+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:21.927 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:21.921+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:21.963 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:21.958+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:22.024 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:22.019+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:22.167 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:22.162+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:22.338 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:22.330+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:22.841 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:22.836+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:22.938 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:22.933+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:23.004 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:22.998+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:23.009 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:23.003+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:23.154 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:23.149+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:23.412 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:23.407+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:23.825 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:23.820+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:23.908 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:23.902+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:23.987 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:23.982+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:24.054 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:24.049+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:24.175 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:24.170+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:24.397 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:24.392+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:24.848 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:24.844+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:24.883 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:24.878+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:24.979 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:24.973+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:25.094 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:25.088+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:25.159 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:25.153+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:25.451 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:25.445+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:25.809 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:25.804+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:25.885 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:25.879+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:26.027 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:26.022+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:26.110 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:26.104+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:26.212 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:26.206+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:26.481 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:26.476+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:26.847 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:26.838+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:26.860 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:26.855+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:27.021 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:27.016+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:27.129 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:27.124+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:27.195 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:27.190+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:27.533 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:27.528+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:27.871 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:27.865+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:27.872 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:27.866+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:27.994 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:27.989+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:28.175 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:28.170+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:28.189 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:28.183+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:28.686 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:28.546+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:28.869 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:28.863+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:28.922 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:28.917+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:28.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:28.961+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:29.169 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:29.162+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:29.189 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:29.183+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:29.612 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:29.607+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:29.878 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:29.872+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:29.917 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:29.911+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:30.005 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:30.001+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:30.197 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:30.192+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:30.201 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:30.195+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:30.609 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:30.604+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:30.891 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:30.886+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:30.916 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:30.910+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:31.026 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:31.021+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:31.222 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:31.216+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:31.239 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:31.233+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:31.637 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:31.632+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:31.883 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:31.877+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:31.911 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:31.906+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:31.956 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:49:31.952+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 40sec e377 v377) 2019-11-25T16:49:32.023 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:32.017+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:32.201 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:32.196+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:32.280 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:32.274+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:32.599 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:32.594+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:32.667 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:49:32.663+0000 16949700 -1 osd.6 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.332672+0000 front 2019-11-25T16:48:13.331037+0000 (oldest deadline 2019-11-25T16:49:31.932759+0000) 2019-11-25T16:49:32.895 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:32.890+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:32.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:32.926+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:32.990 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:32.985+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:33.204 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:33.198+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:33.260 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:33.255+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:33.567 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:33.561+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:33.684 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:49:33.680+0000 16949700 -1 osd.6 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.332672+0000 front 2019-11-25T16:48:13.331037+0000 (oldest deadline 2019-11-25T16:49:31.932759+0000) 2019-11-25T16:49:33.859 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:33.854+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:33.932 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:33.927+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:34.021 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:34.015+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:34.167 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:34.163+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:34.271 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:34.266+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:34.525 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:34.520+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:34.702 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:49:34.696+0000 16949700 -1 osd.6 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.332672+0000 front 2019-11-25T16:48:13.331037+0000 (oldest deadline 2019-11-25T16:49:31.932759+0000) 2019-11-25T16:49:34.833 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:34.827+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:34.889 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:34.884+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:34.981 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:34.977+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:35.125 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:35.120+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:35.314 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:35.309+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:35.518 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:35.512+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:35.681 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:49:35.675+0000 16949700 -1 osd.6 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.332672+0000 front 2019-11-25T16:48:13.331037+0000 (oldest deadline 2019-11-25T16:49:31.932759+0000) 2019-11-25T16:49:35.869 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:35.864+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:35.870 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:35.865+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:35.996 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:35.992+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:36.093 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:36.088+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:49:36.333 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:49:36.329+0000 16949700 -1 osd.5 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.977964+0000 front 2019-11-25T16:48:16.800079+0000 (oldest deadline 2019-11-25T16:48:59.809578+0000) 2019-11-25T16:49:36.573 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:49:36.568+0000 16949700 -1 osd.0 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:17.048446+0000 front 2019-11-25T16:48:17.171736+0000 (oldest deadline 2019-11-25T16:48:56.438257+0000) 2019-11-25T16:49:36.660 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:49:36.655+0000 16949700 -1 osd.6 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.332672+0000 front 2019-11-25T16:48:13.331037+0000 (oldest deadline 2019-11-25T16:49:31.932759+0000) 2019-11-25T16:49:36.871 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:49:36.866+0000 16949700 -1 osd.1 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:16.049878+0000 front 2019-11-25T16:48:16.051161+0000 (oldest deadline 2019-11-25T16:48:57.154406+0000) 2019-11-25T16:49:36.899 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:49:36.895+0000 16949700 -1 osd.7 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.739842+0000 front 2019-11-25T16:48:13.746543+0000 (oldest deadline 2019-11-25T16:48:57.168802+0000) 2019-11-25T16:49:37.008 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:49:37.003+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 40sec e377 v377) 2019-11-25T16:49:37.036 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:49:37.031+0000 16949700 -1 osd.3 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.627981+0000 front 2019-11-25T16:48:17.048041+0000 (oldest deadline 2019-11-25T16:48:57.015078+0000) 2019-11-25T16:49:37.105 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:49:37.098+0000 16949700 -1 osd.2 377 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:48:13.331827+0000 front 2019-11-25T16:48:13.634480+0000 (oldest deadline 2019-11-25T16:48:57.412994+0000) 2019-11-25T16:50:20.578 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:35:25.160 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:20.583 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:35:25.166 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:20.593 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:35:25.199 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:20.594 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:35:25.200 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:20.599 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:35:25.205 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:20.601 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:35:25.206 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:20.742 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:35:25.348 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:20.744 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:35:25.350 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:20.811 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:35:25.394 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:50:49.620 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:49.614+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:50.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:50.659+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:51.632 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:51.628+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:52.686 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:52.681+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:53.685 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:53.680+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:54.698 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:54.693+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:55.740 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:55.734+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:56.750 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:56.745+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:57.710 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:57.705+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:58.679 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:58.674+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:50:59.653 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:50:59.648+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:00.702 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:00.698+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:01.693 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:01.688+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:02.680 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:02.675+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:03.735 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:03.730+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:04.732 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:04.726+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:05.728 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:05.723+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:06.731 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:06.726+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:07.692 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:07.686+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:08.686 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:08.681+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:09.653 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:09.649+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:10.620 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:10.615+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:11.600 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:11.595+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:12.617 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:12.612+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:13.613 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:13.607+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:14.645 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:14.639+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:15.607 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:15.603+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:16.669 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:16.654+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:17.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:17.661+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:18.702 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:18.687+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:18.708 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:18.688+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:19.224 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:19.219+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:19.614 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:19.603+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:19.675 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:19.670+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:19.679 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:19.675+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:20.268 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:20.263+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:20.610 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:20.604+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:20.712 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:20.706+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:20.714 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:20.708+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:20.785 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:20.780+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:20.971 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:20.966+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:21.246 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:21.242+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:21.646 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:21.641+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:21.729 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:21.723+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:21.730 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:21.724+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:21.758 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:21.753+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:21.991 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:21.985+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:22.289 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:22.285+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:22.606 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:22.481+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:22.612 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:22.607+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:22.727 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:22.722+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:22.730 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:22.726+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:22.734 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:22.730+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:22.746 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:22.742+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:23.043 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:23.037+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:23.279 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:23.274+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:23.655 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:23.650+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:23.688 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:23.684+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:23.703 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:23.699+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:23.704 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:23.700+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:23.716 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:23.711+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:23.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:23.761+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:24.044 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:24.039+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:24.316 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:24.312+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:24.611 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:24.607+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:24.689 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:24.685+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:24.707 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:24.702+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:24.708 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:24.704+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:24.721 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:24.716+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:24.761 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:24.748+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:25.117 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:25.109+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:25.366 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:25.363+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:25.662 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:25.657+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:25.669 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:25.665+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:25.673 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:25.670+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:25.717 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:25.712+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:25.723 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:25.718+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:25.753 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:25.748+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:26.082 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:26.077+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:26.377 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:26.371+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:26.681 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:26.677+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:26.683 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:26.680+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:26.685 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:26.680+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:26.700 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:26.696+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:26.716 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:26.711+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:26.760 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:26.754+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:27.108 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:27.104+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:27.339 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:27.333+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:27.648 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:27.643+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:27.676 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:27.672+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:27.684 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:27.677+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:27.685 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:27.680+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:27.732 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:27.727+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:27.958 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:27.740+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:28.142 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:28.137+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:28.338 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:28.333+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:28.605 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:28.600+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:28.710 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:28.704+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:28.711 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:28.706+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:28.715 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:28.710+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:28.724 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:28.719+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:28.739 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:28.734+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:29.180 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:29.175+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:29.359 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:29.353+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:29.587 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:29.582+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:29.680 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:29.675+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:29.701 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:29.696+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:29.704 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:29.699+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:29.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:29.760+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:29.895 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:29.855+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:30.188 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:30.182+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:30.390 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:30.384+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:30.627 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:30.622+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:30.669 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:30.664+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:30.697 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:30.692+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:30.698 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:30.693+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:30.740 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:30.736+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:30.845 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:30.839+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:31.190 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:31.185+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:31.347 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:31.342+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:31.666 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:31.661+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:31.670 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:31.658+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:31.671 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:31.661+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:31.723 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:31.718+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:31.755 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:31.751+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:31.816 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:31.811+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:32.183 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:32.178+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:32.311 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:32.306+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:32.670 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:32.666+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:32.672 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:32.668+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:32.704 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:32.699+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:32.715 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:32.709+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:32.794 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:32.790+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:32.854 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:32.844+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:33.159 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:33.153+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:33.267 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:33.262+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:33.711 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:33.705+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:33.721 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:33.715+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:33.722 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:33.717+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:33.757 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:33.751+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:33.808 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:33.802+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:33.950 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:33.927+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:34.208 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:34.203+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:34.233 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:34.228+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:34.694 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:34.689+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:34.697 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:34.693+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:34.719 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:34.713+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:34.749 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:34.743+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:34.806 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:34.800+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:34.955 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:34.949+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:35.176 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:35.170+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:35.260 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:35.255+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:35.705 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:35.700+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:35.709 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:35.704+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:35.770 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:35.764+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:35.772 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:35.767+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:35.828 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:35.822+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:36.008 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:35.999+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:36.210 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:36.205+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:36.266 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:36.261+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:36.684 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:36.679+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:36.692 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:36.682+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:36.737 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:36.731+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:36.770 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:36.763+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:36.847 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:36.842+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:37.036 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:37.030+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:37.196 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:37.190+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:37.238 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:37.232+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:37.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:37.660+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:37.665 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:37.661+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:37.726 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:37.721+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:37.781 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:37.775+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:37.844 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:37.839+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:38.047 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:38.041+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:38.162 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:38.156+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:38.235 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:38.229+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:38.707 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:38.702+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:38.710 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:38.706+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:38.776 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:38.770+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:38.818 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:38.812+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:38.864 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:38.858+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:39.080 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:39.074+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:39.209 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:39.203+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:39.220 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:39.215+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:39.691 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:39.686+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:39.691 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:39.687+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:39.761 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:39.757+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:39.809 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:39.805+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:39.965 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:39.944+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:40.064 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:40.060+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:40.211 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:40.205+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:40.223 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:40.218+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:40.719 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:40.714+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:40.722 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:40.717+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:40.781 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:40.776+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:40.787 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:40.783+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:40.912 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:40.906+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:41.099 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:41.094+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:41.216 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:41.211+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:41.249 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:41.244+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:41.703 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:41.697+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:41.703 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:41.699+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:41.761 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:41.756+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:41.765 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:41.760+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:41.883 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:41.877+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:42.116 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:42.112+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:42.205 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:42.200+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:42.213 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:42.208+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:42.686 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:42.679+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:42.686 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:42.681+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:42.718 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:42.713+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:42.786 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:42.777+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:42.915 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:42.908+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:43.075 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:43.070+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:43.166 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:43.161+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:43.223 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:43.218+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:43.718 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:43.713+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:43.753 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:43.746+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:43.754 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:43.749+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:43.780 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:43.775+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:43.920 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:43.915+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:44.046 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:44.042+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:44.206 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:44.200+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:44.279 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:44.275+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:44.775 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:44.769+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:44.801 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:44.796+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:44.802 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:44.797+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:44.833 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:44.828+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:44.953 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:44.947+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:45.090 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:45.085+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:45.215 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:45.209+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:45.316 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:45.311+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:45.744 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:45.740+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:45.838 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:45.832+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:45.838 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:45.833+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:45.855 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:45.851+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:46.046 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:46.040+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:46.115 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:46.111+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:46.226 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:46.222+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:46.310 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:46.306+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:46.787 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:46.782+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:46.801 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:46.795+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:46.801 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:46.796+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:46.891 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:46.887+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:47.152 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:47.148+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:47.209 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:47.202+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:47.226 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:47.221+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:47.268 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:47.264+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:47.757 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:47.750+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:47.757 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:47.752+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:47.829 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:47.825+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:47.904 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:47.899+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:48.152 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:48.146+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:48.226 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:48.221+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:48.230 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:48.226+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:48.320 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:48.312+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:48.738 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:48.732+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:48.739 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:48.733+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:48.802 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:48.797+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:48.870 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:48.865+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:49.109 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:49.104+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:49.246 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:49.241+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:49.277 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:49.272+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:49.328 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:49.323+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:49.705 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:49.699+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:49.706 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:49.701+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:49.818 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:49.813+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:49.921 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:49.916+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:50.102 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:50.093+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:50.268 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:50.263+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:50.292 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:50.277+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:50.300 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:50.295+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:50.752 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:50.746+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:50.753 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:50.747+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:50.829 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:50.825+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:50.881 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:50.876+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:51.108 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:51.103+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:51.254 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:51.248+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:51.305 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:51.299+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:51.472 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:51:51.466+0000 16949700 -1 osd.5 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.477813+0000 front 2019-11-25T16:50:37.880917+0000 (oldest deadline 2019-11-25T16:51:21.936383+0000) 2019-11-25T16:51:51.774 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:51.764+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:51.779 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:51.773+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:51.858 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:51:51.854+0000 16949700 -1 osd.3 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:37.464682+0000 front 2019-11-25T16:50:37.497572+0000 (oldest deadline 2019-11-25T16:51:19.918099+0000) 2019-11-25T16:51:51.889 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:51:51.882+0000 16949700 -1 osd.2 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.837623+0000 front 2019-11-25T16:50:34.831025+0000 (oldest deadline 2019-11-25T16:51:19.577945+0000) 2019-11-25T16:51:52.138 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:51:52.134+0000 16949700 -1 osd.1 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:38.111181+0000 front 2019-11-25T16:50:38.550461+0000 (oldest deadline 2019-11-25T16:51:21.817507+0000) 2019-11-25T16:51:52.278 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:51:52.273+0000 16949700 -1 osd.7 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.926636+0000 front 2019-11-25T16:50:36.809868+0000 (oldest deadline 2019-11-25T16:51:18.178580+0000) 2019-11-25T16:51:52.311 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:51:52.306+0000 16949700 -1 osd.0 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:35.148189+0000 front 2019-11-25T16:50:35.131819+0000 (oldest deadline 2019-11-25T16:51:20.904256+0000) 2019-11-25T16:51:52.440 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:51:52.434+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 45sec e382 v382) 2019-11-25T16:51:52.837 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:52.819+0000 16949700 -1 osd.4 382 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:50:33.342048+0000 front 2019-11-25T16:50:33.339158+0000 (oldest deadline 2019-11-25T16:51:18.226468+0000) 2019-11-25T16:51:52.838 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:52.820+0000 16949700 -1 osd.4 382 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=25 ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:53.902 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:53.889+0000 16949700 -1 osd.4 383 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=25 ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:55.476 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:55.162+0000 16949700 -1 osd.4 384 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=25 ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:56.249 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:56.241+0000 16949700 -1 osd.4 384 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=25 ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:51:57.694 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:51:57.586+0000 16949700 -1 osd.4 384 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=25 ondisk+retry+read+known_if_redirected e381) 2019-11-25T16:52:48.681 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:48.676+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:49.107 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:49.101+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:49.147 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:49.142+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:49.454 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:49.448+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:49.627 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:49.621+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:49.647 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:49.642+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:49.917 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:49.912+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:50.109 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:50.104+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:50.155 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:50.150+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:50.417 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:50.412+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:50.641 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:50.635+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:50.697 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:50.692+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:50.915 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:50.910+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:51.146 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:51.141+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:51.175 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:51.171+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:51.229 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:51.218+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:51.377 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:51.371+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:51.619 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:51.614+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:51.700 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:51.695+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:51.959 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:51.952+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:52.193 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:52.187+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:52.201 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:52.196+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:52.210 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:52.204+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:52.360 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:52.355+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:52.574 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:52.569+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:52.658 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:52.650+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:52.966 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:52.960+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:53.155 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:53.150+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:53.223 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:53.218+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:53.244 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:53.240+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:53.413 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:53.407+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:53.627 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:53.621+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:53.673 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:53.669+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:53.994 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:53.989+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:54.150 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:54.144+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:54.258 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:54.253+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:54.292 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:54.287+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:54.394 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:54.388+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:54.610 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:54.603+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:54.703 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:54.698+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:55.032 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:55.025+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:55.118 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:55.113+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:55.286 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:55.281+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:55.310 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:55.306+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:55.441 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:55.437+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:55.570 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:55.565+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:55.678 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:55.672+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:56.000 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:55.996+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:56.102 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:56.098+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:56.323 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:56.318+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:56.324 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:56.320+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:56.442 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:56.437+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:56.584 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:56.579+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:56.727 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:56.722+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:56.987 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:56.982+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:57.056 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:57.052+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:57.343 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:57.338+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:57.355 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:57.349+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:57.465 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:57.460+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:57.592 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:57.587+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:57.773 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:57.768+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:57.944 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:57.939+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:58.051 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:58.044+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:58.372 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:58.367+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:58.398 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:58.392+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:58.468 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:58.463+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:58.606 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:58.600+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:58.758 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:58.752+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:58.992 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:58.987+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:52:59.053 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:52:59.048+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:52:59.349 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:52:59.344+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:52:59.412 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:52:59.407+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:52:59.505 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:52:59.501+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:52:59.641 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:52:59.637+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:52:59.724 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:52:59.719+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:52:59.952 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:52:59.948+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:00.039 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:00.034+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:00.383 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:00.377+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:00.465 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:00.461+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:00.469 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:00.464+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:00.598 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:00.592+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:00.735 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:00.730+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:00.955 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:00.951+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:01.047 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:01.042+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:01.425 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:01.420+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:01.442 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:01.435+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:01.526 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:01.521+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:01.573 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:01.569+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:01.781 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:01.776+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:01.929 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:01.924+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:02.059 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:02.053+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:02.405 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:02.400+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:02.439 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:02.433+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:02.554 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:02.550+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:02.608 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:02.600+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:02.756 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:02.751+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:02.900 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:02.895+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:03.040 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:03.034+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:03.382 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:03.377+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:03.395 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:03.390+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:03.579 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:03.574+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:03.593 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:03.588+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:03.807 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:03.802+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:03.863 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:03.858+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:04.039 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:04.034+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:04.381 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:04.377+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:04.402 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:04.397+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:04.555 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:04.551+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:04.597 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:04.593+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:04.854 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:04.849+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:04.865 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:04.860+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:04.998 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:04.993+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:05.354 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:05.349+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:05.368 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:05.363+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:05.614 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:05.607+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:05.620 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:05.616+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:05.867 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:05.863+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:05.914 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:05.909+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:05.994 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:05.988+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:06.339 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:06.334+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:06.347 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:06.341+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:06.581 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:06.576+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:06.661 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:06.657+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:06.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:06.909+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:06.932 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:06.928+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:06.965 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:06.959+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:07.334 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:07.328+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:07.350 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:07.345+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:07.629 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:07.624+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:07.717 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:07.712+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:07.889 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:07.885+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:07.908 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:07.902+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:08.016 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:08.010+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:08.341 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:08.336+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:08.374 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:08.370+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:08.681 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:08.674+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:08.708 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:08.702+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:08.884 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:08.879+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:08.938 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:08.933+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:08.988 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:08.982+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:09.311 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:09.306+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:09.376 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:09.371+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:09.683 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:09.677+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:09.720 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:09.713+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:09.845 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:09.839+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:09.936 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:09.931+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:09.973 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:09.968+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:10.333 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:10.328+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:10.407 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:10.402+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:10.667 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:10.661+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:10.681 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:10.674+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:10.882 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:10.877+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:10.955 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:10.950+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:10.988 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:10.983+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:11.292 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:11.286+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:11.449 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:11.443+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:11.687 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:11.682+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:11.689 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:11.683+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:11.848 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:11.842+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:11.985 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:11.980+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:12.006 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:12.001+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:12.270 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:12.265+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:12.436 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:12.430+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:12.668 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:12.663+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:12.703 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:12.697+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:12.857 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:12.851+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:12.941 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:12.936+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:13.024 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:13.019+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:13.273 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:13.267+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:13.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:13.442+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:13.652 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:13.646+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:13.723 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:13.718+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:13.842 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:13.834+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:13.973 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:13.968+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:14.053 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:14.048+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:14.326 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:14.320+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:14.428 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:14.423+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:14.698 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:14.692+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:14.710 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:14.705+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:14.867 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:14.860+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:14.970 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:14.965+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:15.054 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:15.042+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:15.337 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:15.331+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:15.480 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:15.474+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:15.694 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:15.688+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:15.733 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:15.728+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:15.857 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:15.852+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:15.997 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:15.992+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:16.013 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:16.008+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:16.352 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:16.347+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:16.456 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:16.449+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:16.700 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:16.695+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:16.709 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:16.703+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:16.874 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:16.868+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:16.953 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:16.948+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:16.993 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:16.989+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:17.351 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:17.346+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:17.418 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:17.412+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:17.708 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:17.701+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:17.737 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:17.732+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:17.901 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:17.894+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:17.988 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:17.982+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:17.996 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:17.992+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:18.341 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:18.336+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:18.469 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:18.463+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:18.684 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:18.678+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:18.741 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:18.736+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:18.944 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:18.938+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:18.948 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:18.942+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:19.026 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:19.020+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:19.358 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:19.351+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:19.503 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:19.497+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:19.667 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:19.662+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:19.722 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:19.716+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:19.937 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:19.930+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:19.952 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:19.946+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:20.045 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:20.040+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:20.355 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:20.348+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:20.517 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:20.511+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:20.681 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:20.675+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:20.725 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:20.718+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:20.953 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:20.946+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:20.959 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:20.952+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:21.046 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:21.040+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:21.316 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:21.310+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:21.551 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:21.545+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:21.735 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:21.730+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:21.755 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:21.748+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:21.921 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:21.915+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:21.979 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:21.971+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:22.087 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:22.082+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:22.312 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:22.306+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:22.597 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:22.591+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:22.724 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:22.718+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:22.743 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:53:22.738+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 46sec e388 v388) 2019-11-25T16:53:22.764 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:22.758+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:22.956 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:22.950+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:22.990 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:22.984+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:23.047 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:23.040+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:23.358 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:23.351+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:23.629 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:23.623+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:23.734 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:23.728+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:23.773 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:23.767+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:23.969 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:23.964+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:24.000 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:23.995+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:24.003 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:23.997+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:24.400 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:24.394+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:24.669 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:24.662+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:24.771 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:24.765+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:24.777 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:24.771+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:24.959 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:24.952+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:25.003 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:24.997+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:25.022 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:25.016+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:25.411 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:25.406+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:25.686 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:25.681+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:25.727 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:25.720+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:25.807 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:25.802+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:25.999 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:25.995+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:26.006 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:26.002+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:26.053 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:26.048+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:26.450 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:26.445+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:26.706 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:26.702+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:26.768 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:26.762+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:26.788 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:26.782+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:26.986 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:26.981+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:53:27.037 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:53:27.032+0000 16949700 -1 osd.3 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:03.998465+0000 front 2019-11-25T16:52:03.737900+0000 (oldest deadline 2019-11-25T16:52:48.372228+0000) 2019-11-25T16:53:27.102 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:53:27.097+0000 16949700 -1 osd.5 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.655746+0000 front 2019-11-25T16:52:06.662975+0000 (oldest deadline 2019-11-25T16:52:49.447398+0000) 2019-11-25T16:53:27.422 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:53:27.417+0000 16949700 -1 osd.2 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:06.920323+0000 front 2019-11-25T16:52:06.748015+0000 (oldest deadline 2019-11-25T16:52:50.833668+0000) 2019-11-25T16:53:27.720 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:53:27.715+0000 16949700 -1 osd.0 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:04.827945+0000 front 2019-11-25T16:52:04.831807+0000 (oldest deadline 2019-11-25T16:52:48.902594+0000) 2019-11-25T16:53:27.782 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:53:27.777+0000 16949700 -1 osd.7 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.677712+0000 front 2019-11-25T16:52:05.726115+0000 (oldest deadline 2019-11-25T16:52:49.159513+0000) 2019-11-25T16:53:27.827 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:53:27.821+0000 16949700 -1 osd.6 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.657370+0000 front 2019-11-25T16:52:05.907579+0000 (oldest deadline 2019-11-25T16:52:49.073313+0000) 2019-11-25T16:53:27.877 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:53:27.870+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 46sec e388 v388) 2019-11-25T16:53:27.997 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:53:27.991+0000 16949700 -1 osd.1 388 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:52:05.585596+0000 front 2019-11-25T16:52:05.574707+0000 (oldest deadline 2019-11-25T16:52:48.675061+0000) 2019-11-25T16:54:12.065 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:39:16.647 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:54:12.080 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:39:16.663 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:54:12.092 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:39:16.698 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:54:12.098 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:39:16.704 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:54:12.099 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:39:16.705 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:54:12.105 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:39:16.711 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:54:12.340 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:39:16.946 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:54:12.344 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:39:16.950 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:54:12.395 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:39:16.978 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:55:02.823 INFO:teuthology.orchestra.run.smithi153.stderr:testIfMatch (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-11-25T16:55:05.664 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:05.658+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:06.720 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:06.712+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:07.465 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:07.459+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:07.718 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:07.712+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:08.177 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:08.171+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:08.324 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:08.317+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:08.467 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:08.459+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:08.696 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:08.690+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:08.742 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:08.737+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:09.228 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:09.220+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:09.378 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:09.372+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:09.468 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:09.463+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:09.652 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:09.644+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:09.688 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:09.682+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:09.714 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:09.708+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:09.816 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:09.811+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:10.258 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:10.251+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:10.386 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:10.372+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:10.464 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:10.458+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:10.684 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:10.677+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:10.690 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:10.683+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:10.704 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:10.698+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:10.785 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:10.778+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:11.300 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:11.295+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:11.368 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:11.363+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:11.458 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:11.451+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:11.732 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:11.727+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:11.737 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:11.731+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:11.739 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:11.734+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:11.834 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:11.828+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:12.315 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:12.310+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:12.376 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:12.371+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:12.481 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:12.476+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:12.748 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:12.742+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:12.752 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:12.747+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:12.758 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:12.751+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:12.788 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:12.783+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:13.303 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:13.298+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:13.422 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:13.417+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:13.513 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:13.507+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:13.775 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:13.768+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:13.779 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:13.773+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:13.780 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:13.775+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:13.811 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:13.804+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:14.278 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:14.274+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:14.399 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:14.393+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:14.567 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:14.561+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:14.757 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:14.752+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:14.769 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:14.765+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:14.776 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:14.771+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:14.798 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:14.792+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:15.299 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:15.294+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:15.400 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:15.395+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:15.563 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:15.557+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:15.765 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:15.759+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:15.780 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:15.775+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:15.796 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:15.791+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:15.813 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:15.808+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:16.346 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:16.342+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:16.413 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:16.409+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:16.599 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:16.593+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:16.742 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:16.736+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:16.750 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:16.744+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:16.815 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:16.809+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:16.818 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:16.813+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:17.302 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:17.296+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:17.378 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:17.371+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:17.607 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:17.601+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:17.707 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:17.701+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:17.753 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:17.747+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:17.800 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:17.794+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:17.859 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:17.854+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:18.342 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:18.337+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:18.427 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:18.422+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:18.670 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:18.658+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:18.749 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:18.744+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:18.763 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:18.758+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:18.801 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:18.794+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:18.893 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:18.889+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:19.302 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:19.297+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:19.394 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:19.390+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:19.705 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:19.698+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:19.711 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:19.705+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:19.790 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:19.786+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:19.820 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:19.813+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:19.874 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:19.868+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:20.325 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:20.320+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:20.421 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:20.416+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:20.691 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:20.685+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:20.742 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:20.736+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:20.759 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:20.753+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:20.845 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:20.839+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:20.868 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:20.862+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:21.302 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:21.297+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:21.397 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:21.391+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:21.677 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:21.671+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:21.740 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:21.734+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:21.749 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:21.744+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:21.829 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:21.823+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:21.853 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:21.847+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:22.290 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:22.285+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:22.370 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:22.364+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:22.719 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:22.713+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:22.736 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:22.731+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:22.767 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:22.761+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:22.814 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:22.809+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:22.877 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:22.872+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:23.285 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:23.280+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:23.358 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:23.352+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:23.737 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:23.732+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:23.747 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:23.740+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:23.753 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:23.748+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:23.858 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:23.852+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:23.926 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:23.922+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:24.256 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:24.250+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:24.384 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:24.380+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:24.717 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:24.712+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:24.732 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:24.727+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:24.738 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:24.734+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:24.873 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:24.867+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:24.924 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:24.919+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:25.243 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:25.238+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:25.371 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:25.367+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:25.688 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:25.682+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:25.693 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:25.688+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:25.748 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:25.742+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:25.901 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:25.895+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:25.919 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:25.913+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:26.230 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:26.226+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:26.354 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:26.349+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:26.731 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:26.725+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:26.734 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:26.729+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:26.745 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:26.739+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:26.906 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:26.902+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:26.922 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:26.915+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:27.278 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:27.273+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:27.352 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:27.348+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:27.716 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:27.710+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:27.739 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:27.735+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:27.843 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:27.838+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:27.879 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:27.876+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:27.914 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:27.908+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:28.334 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:28.329+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:28.343 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:28.335+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:28.714 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:28.710+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:28.750 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:28.745+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:28.881 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:28.844+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:28.891 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:28.886+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:28.932 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:28.927+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:29.305 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:29.301+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:29.360 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:29.355+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:29.723 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:29.719+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:29.765 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:29.761+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:29.874 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:29.869+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:29.875 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:29.870+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:29.930 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:29.925+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:30.283 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:30.279+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:30.348 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:30.342+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:30.698 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:30.694+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:30.798 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:30.794+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:30.870 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:30.866+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:30.893 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:30.888+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:30.967 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:30.963+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:31.293 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:31.289+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:31.362 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:31.357+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:31.679 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:31.674+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:31.758 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:31.754+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:31.879 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:31.874+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:31.943 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:31.938+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:31.990 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:31.984+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:32.268 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:32.263+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:32.417 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:32.412+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:32.696 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:32.692+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:32.744 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:32.739+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:32.867 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:32.863+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:32.922 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:32.918+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:32.991 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:32.985+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:33.234 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:33.229+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:33.408 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:33.403+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:33.708 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:33.703+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:33.738 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:33.733+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:33.848 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:33.844+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:33.967 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:33.963+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:33.999 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:33.994+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:34.275 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:34.270+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:34.444 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:34.439+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:34.673 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:34.668+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:34.776 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:34.772+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:34.851 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:34.846+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:34.943 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:34.938+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:34.997 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:34.991+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:35.248 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:35.242+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:35.458 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:35.452+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:35.720 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:35.716+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:35.795 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:35.790+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:35.844 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:35.839+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:35.919 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:35.915+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:36.007 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:36.003+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:36.282 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:36.276+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:36.484 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:36.478+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:36.743 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:36.738+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:36.819 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:36.815+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:36.851 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:36.847+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:36.882 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:36.877+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:36.981 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:36.977+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:37.308 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:37.303+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:37.502 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:37.497+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:37.772 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:37.767+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:37.783 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:37.779+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:37.846 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:37.841+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:37.921 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:37.916+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:38.013 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:38.008+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:38.317 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:38.311+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:38.498 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:55:38.494+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 44sec e394 v394) 2019-11-25T16:55:38.526 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:38.521+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:38.781 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:38.777+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:38.818 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:38.813+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:38.845 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:38.840+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:38.896 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:38.892+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:38.980 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:38.975+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:39.323 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:39.319+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:39.550 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:39.545+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:39.760 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:39.756+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:39.805 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:39.799+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:39.881 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:39.876+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:39.907 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:39.902+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:39.989 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:39.984+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:40.361 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:40.357+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:40.570 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:40.564+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:40.784 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:40.779+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:40.803 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:40.798+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:40.861 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:40.856+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:40.895 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:40.889+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:41.000 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:40.995+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:41.399 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:41.394+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:41.581 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:41.577+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:41.773 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:41.768+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:41.816 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:41.810+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:41.863 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:41.857+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:41.924 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:41.912+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:42.051 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:42.047+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:42.410 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:42.404+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:42.565 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:42.559+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:42.796 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:42.790+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:42.827 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:42.822+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:42.848 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:42.843+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:42.900 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:42.895+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:43.072 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:43.067+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:43.446 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:43.441+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:43.507 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:55:43.502+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 44sec e394 v394) 2019-11-25T16:55:43.532 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:43.528+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:43.750 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:43.746+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:43.826 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:43.821+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:43.829 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:43.824+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:43.893 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:43.888+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:44.047 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:44.040+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:44.468 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:44.463+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:44.518 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:44.512+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:44.802 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:44.796+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:44.819 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:44.814+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:44.840 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:44.835+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:44.871 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:44.866+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:45.008 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:45.002+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:45.497 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:45.491+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:45.556 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:45.551+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:45.829 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:45.822+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:45.856 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:45.850+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:45.888 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:45.883+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:45.896 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:45.890+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:46.032 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:46.028+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:46.514 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:46.509+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:46.569 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:46.564+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:46.785 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:46.780+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:46.836 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:46.830+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:46.894 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:46.886+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:46.906 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:46.901+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:47.079 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:47.074+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:47.516 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:47.511+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:47.544 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:47.539+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:55:47.767 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:55:47.762+0000 16949700 -1 osd.2 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.383710+0000 front 2019-11-25T16:54:24.369276+0000 (oldest deadline 2019-11-25T16:55:08.402958+0000) 2019-11-25T16:55:47.798 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:55:47.792+0000 16949700 -1 osd.6 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.928527+0000 front 2019-11-25T16:54:25.966562+0000 (oldest deadline 2019-11-25T16:55:09.776827+0000) 2019-11-25T16:55:47.882 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:55:47.877+0000 16949700 -1 osd.5 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.136063+0000 front 2019-11-25T16:54:22.827972+0000 (oldest deadline 2019-11-25T16:55:06.806411+0000) 2019-11-25T16:55:47.943 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:55:47.937+0000 16949700 -1 osd.7 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:22.513033+0000 front 2019-11-25T16:54:25.263378+0000 (oldest deadline 2019-11-25T16:55:04.933317+0000) 2019-11-25T16:55:48.076 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:55:48.070+0000 16949700 -1 osd.0 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:24.860500+0000 front 2019-11-25T16:54:24.876875+0000 (oldest deadline 2019-11-25T16:55:09.208283+0000) 2019-11-25T16:55:48.567 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:55:48.561+0000 16949700 -1 osd.3 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:23.684652+0000 front 2019-11-25T16:54:25.963677+0000 (oldest deadline 2019-11-25T16:55:07.380504+0000) 2019-11-25T16:55:48.583 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T16:55:48.576+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 44sec e394 v394) 2019-11-25T16:55:48.592 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:55:48.587+0000 16949700 -1 osd.1 394 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:54:26.096523+0000 front 2019-11-25T16:54:24.067275+0000 (oldest deadline 2019-11-25T16:55:07.503596+0000) 2019-11-25T16:56:18.787 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:56:18.779+0000 16949700 -1 osd.2 396 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120160 5.0 5:05bf5b68:::notify.1:head [watch reconnect cookie 821195984 gen 97] snapc 0=[] ondisk+write+known_if_redirected e395) 2019-11-25T16:56:18.883 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:41:23.466 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:18.892 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:41:23.475 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:18.917 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:41:23.522 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:18.924 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:41:23.529 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:18.954 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:41:23.560 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:18.973 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:41:23.579 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:19.032 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:41:23.615 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:19.087 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:56:19.084+0000 16949700 -1 osd.6 396 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e395) 2019-11-25T16:56:19.138 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:41:23.744 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:19.157 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:41:23.762 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:56:20.306 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:56:20.167+0000 16949700 -1 osd.6 397 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e395) 2019-11-25T16:56:21.524 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:56:21.432+0000 16949700 -1 osd.6 397 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=29 ondisk+retry+read+known_if_redirected e395) 2019-11-25T16:56:37.084 INFO:teuthology.orchestra.run.smithi153.stderr:testIfMatchAndUnmodified (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-11-25T16:56:55.212 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:56:55.207+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:56:56.238 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:56:56.233+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:56:57.260 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:56:57.255+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:56:58.303 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:56:58.298+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:56:59.333 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:56:59.327+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:00.314 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:00.304+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:01.300 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:01.295+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:02.256 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:02.252+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:03.275 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:03.270+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:04.288 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:04.284+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:05.314 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:05.309+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:06.297 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:06.292+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:07.314 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:07.309+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:08.339 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:08.335+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:09.344 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:09.339+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:10.340 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:10.336+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:11.305 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:11.301+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:12.322 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:12.319+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:13.323 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:13.320+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:14.324 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:14.320+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:15.284 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:15.281+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:16.303 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:16.299+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:16.451 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:16.446+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:17.206 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:17.201+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:17.351 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:17.346+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:17.475 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:17.469+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:18.166 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:18.162+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:18.342 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:18.337+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:18.361 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:18.356+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:18.362 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:18.358+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:18.428 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:18.420+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:18.504 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:18.500+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:19.070 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:19.064+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:19.172 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:19.168+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:19.330 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:19.326+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:19.381 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:19.376+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:19.382 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:19.377+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:19.466 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:19.462+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:19.482 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:19.476+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:20.084 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:20.080+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:20.200 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:20.195+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:20.352 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:20.347+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:20.382 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:20.378+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:20.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:20.381+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:20.439 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:20.436+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:20.450 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:20.446+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:21.044 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:21.036+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:21.067 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:21.063+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:21.238 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:21.234+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:21.344 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:21.339+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:21.347 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:21.342+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:21.381 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:21.375+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:21.452 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:21.446+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:21.475 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:21.470+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:22.049 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:22.044+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:22.122 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:22.116+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:22.280 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:22.274+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:22.395 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:22.388+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:22.396 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:22.390+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:22.416 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:22.411+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:22.456 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:22.452+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:22.460 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:22.454+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:23.022 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:23.017+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:23.150 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:23.145+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:23.238 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:23.233+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:23.384 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:23.378+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:23.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:23.379+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:23.443 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:23.438+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:23.461 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:23.456+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:23.464 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:23.460+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:23.987 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:23.983+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:24.116 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:24.112+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:24.271 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:24.266+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:24.437 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:24.430+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:24.438 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:24.432+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:24.445 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:24.441+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:24.463 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:24.459+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:24.500 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:24.496+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:25.026 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:25.020+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:25.161 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:25.157+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:25.371 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:25.365+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:25.402 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:25.397+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:25.406 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:25.401+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:25.420 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:25.415+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:25.480 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:25.476+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:25.547 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:25.542+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:26.058 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:26.053+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:26.138 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:26.134+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:26.338 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:26.332+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:26.388 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:26.383+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:26.415 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:26.411+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:26.417 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:26.413+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:26.530 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:26.525+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:26.542 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:26.536+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:27.026 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:27.021+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:27.150 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:27.145+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:27.374 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:27.369+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:27.424 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:27.419+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:27.450 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:27.444+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:27.450 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:27.445+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:27.545 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:27.540+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:27.556 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:27.551+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:28.015 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:28.010+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:28.219 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:28.205+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:28.405 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:28.400+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:28.408 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:28.402+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:28.437 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:28.432+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:28.438 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:28.433+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:28.569 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:28.563+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:28.580 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:28.573+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:29.057 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:29.051+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:29.201 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:29.192+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:29.418 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:29.411+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:29.418 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:29.413+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:29.425 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:29.419+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:29.427 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:29.422+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:29.584 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:29.580+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:29.610 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:29.606+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:30.095 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:30.090+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:30.162 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:30.156+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:30.395 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:30.389+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:30.401 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:30.396+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:30.496 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:30.491+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:30.497 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:30.492+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:30.552 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:30.546+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:30.618 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:30.614+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:31.087 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:31.082+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:31.116 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:31.111+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:31.442 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:31.436+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:31.459 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:31.453+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:31.522 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:31.516+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:31.525 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:31.520+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:31.589 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:31.584+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:31.614 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:31.610+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:32.097 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:32.093+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:32.152 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:32.148+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:32.401 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:32.395+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:32.506 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:32.501+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:32.563 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:32.558+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:32.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:32.559+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:32.599 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:32.593+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:32.624 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:32.619+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:33.088 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:33.083+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:33.190 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:33.184+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:33.405 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:33.399+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:33.550 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:33.545+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:33.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:33.557+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:33.565 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:33.560+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:33.590 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:33.585+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:33.639 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:33.634+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:34.072 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:34.067+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:34.235 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:34.231+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:34.359 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:34.354+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:34.529 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:34.522+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:34.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:34.558+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:34.565 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:34.559+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:34.579 INFO:teuthology.orchestra.run.smithi153.stderr:testIfModifiedSince (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-11-25T16:57:34.580 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:34.575+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:34.684 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:34.679+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:35.063 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:35.054+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:35.200 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:35.195+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:35.374 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:35.368+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:35.491 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:35.486+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:35.601 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:35.589+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:35.604 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:35.599+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:35.614 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:35.591+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:35.679 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:35.675+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:36.019 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:36.014+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:36.183 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:36.178+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:36.339 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:36.334+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:36.535 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:36.529+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:36.576 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:36.569+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:36.577 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:36.571+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:36.612 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:36.604+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:36.690 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:36.685+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:36.989 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:36.984+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:37.183 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:37.177+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:37.335 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:37.330+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:37.505 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:37.499+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:37.556 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:37.550+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:37.557 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:37.552+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:37.580 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:37.576+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:37.731 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:37.726+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:38.017 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:38.012+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:38.179 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:38.175+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:38.343 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:38.338+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:38.467 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:38.461+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:38.526 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:38.519+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:38.527 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:38.521+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:38.563 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:38.558+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:38.721 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:38.716+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:39.015 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:39.010+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:39.176 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:39.172+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:39.365 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:39.361+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:39.492 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:39.486+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:39.532 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:39.526+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:39.533 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:39.528+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:39.556 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:39.551+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:39.716 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:39.711+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:40.015 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:40.010+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:40.195 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:40.190+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:40.395 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:40.390+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:40.515 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:40.509+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:40.515 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:40.511+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:40.537 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:40.532+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:40.546 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:40.540+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:40.699 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:40.694+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:41.056 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:41.051+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:41.172 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:41.167+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:41.371 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:41.365+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:41.538 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:41.532+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:41.541 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:41.536+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:41.560 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:41.555+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:41.588 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:41.582+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:41.728 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:41.724+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:42.050 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:42.045+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:42.198 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:42.193+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:42.405 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:42.400+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:42.510 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:42.505+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:42.513 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:42.508+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:42.601 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:42.597+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:42.613 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:42.609+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:42.781 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:42.776+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:43.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:43.095+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:43.232 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:43.227+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:43.362 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:43.357+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:43.529 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:43.524+0000 16949700 -1 osd.4 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:35.089877+0000 front 2019-11-25T16:56:34.992148+0000 (oldest deadline 2019-11-25T16:57:18.189796+0000) 2019-11-25T16:57:43.537 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T16:57:43.528+0000 16949700 -1 osd.4 400 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e398) 2019-11-25T16:57:43.629 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:43.624+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:43.635 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:57:43.628+0000 16949700 -1 osd.5 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.687769+0000 front 2019-11-25T16:56:33.680574+0000 (oldest deadline 2019-11-25T16:57:16.333345+0000) 2019-11-25T16:57:43.768 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:43.763+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:57:44.095 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:57:44.089+0000 16949700 -1 osd.0 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.411034+0000 front 2019-11-25T16:56:34.316025+0000 (oldest deadline 2019-11-25T16:57:18.365299+0000) 2019-11-25T16:57:44.251 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:57:44.246+0000 16949700 -1 osd.3 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.943293+0000 front 2019-11-25T16:56:34.998713+0000 (oldest deadline 2019-11-25T16:57:20.818686+0000) 2019-11-25T16:57:44.399 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:57:44.394+0000 16949700 -1 osd.7 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:31.745048+0000 front 2019-11-25T16:56:31.913025+0000 (oldest deadline 2019-11-25T16:57:15.825617+0000) 2019-11-25T16:57:44.630 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:57:44.625+0000 16949700 -1 osd.1 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:33.768328+0000 front 2019-11-25T16:56:33.684594+0000 (oldest deadline 2019-11-25T16:57:18.352970+0000) 2019-11-25T16:57:44.736 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:57:44.731+0000 16949700 -1 osd.2 400 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T16:56:34.693407+0000 front 2019-11-25T16:56:34.695427+0000 (oldest deadline 2019-11-25T16:57:17.924701+0000) 2019-11-25T16:58:16.372 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:43:20.955 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:16.376 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:43:20.959 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:16.386 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:43:20.991 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:16.387 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:43:20.993 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:16.390 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:43:20.996 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:16.396 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:43:21.002 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:16.524 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:43:21.106 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:16.533 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:43:21.139 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:16.541 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:43:21.147 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T16:58:33.081 INFO:teuthology.orchestra.run.smithi153.stderr:testIfNoneMatch (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-11-25T16:59:21.706 INFO:teuthology.orchestra.run.smithi153.stderr:testIfUnmodifiedSince (test.functional.tests.TestFileComparisonUTF8) ... ok 2019-11-25T16:59:35.333 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:35.328+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:35.681 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:35.675+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:36.098 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:36.092+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:36.288 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:36.283+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:36.654 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:36.649+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:37.098 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:37.093+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:37.135 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:37.131+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:37.261 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:37.257+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:37.309 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:37.304+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:37.623 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:37.617+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:38.123 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:38.117+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:38.138 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:38.131+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:38.249 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:38.243+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:38.353 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:38.347+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:38.646 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:38.641+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:39.093 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:39.087+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:39.127 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:39.121+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:39.169 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:39.165+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:39.215 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:39.209+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:39.366 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:39.359+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:39.560 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:39.554+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:39.669 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:39.664+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:40.101 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:40.094+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:40.160 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:40.154+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:40.182 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:40.177+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:40.257 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:40.251+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:40.350 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:40.345+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:40.593 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:40.588+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:40.678 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:40.673+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:41.080 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:41.075+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:41.145 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:41.141+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:41.162 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:41.156+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:41.298 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:41.293+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:41.379 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:41.374+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:41.583 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:41.578+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:41.634 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:41.629+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:42.114 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:42.109+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:42.122 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:42.118+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:42.145 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:42.141+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:42.297 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:42.283+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:42.346 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:42.342+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:42.556 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:42.551+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:42.631 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:42.626+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:43.116 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:43.112+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:43.131 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:43.126+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:43.148 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:43.142+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:43.337 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:43.332+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:43.362 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:43.357+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:43.518 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:43.512+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:43.670 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:43.665+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:44.113 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:44.109+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:44.154 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:44.149+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:44.173 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:44.168+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:44.324 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:44.320+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:44.331 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:44.326+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:44.571 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:44.565+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:44.632 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:44.626+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:45.082 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:45.077+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:45.186 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:45.180+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:45.207 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:45.202+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:45.365 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:45.359+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:45.376 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:45.371+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:45.590 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:45.585+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:45.591 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:45.585+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:46.096 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:46.091+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:46.176 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:46.170+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:46.186 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:46.182+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:46.338 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:46.333+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:46.388 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:46.384+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:46.568 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:46.563+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:46.702 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:46.695+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:47.130 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:47.125+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:47.187 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:47.183+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:47.206 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:47.200+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:47.341 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:47.337+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:47.387 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:47.381+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:47.590 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:47.584+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:47.660 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:47.654+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:48.149 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:48.143+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:48.154 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:48.149+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:48.256 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:48.252+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:48.326 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:48.321+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:48.406 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:48.401+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:48.592 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:48.586+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:48.701 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:48.696+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:49.119 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:49.114+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:49.168 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:49.162+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:49.297 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:49.292+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:49.327 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:49.322+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:49.412 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:49.407+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:49.587 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:49.582+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:49.710 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:49.706+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:50.104 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:50.099+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:50.203 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:50.198+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:50.317 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:50.312+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:50.333 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:50.328+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:50.457 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:50.452+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:50.619 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:50.614+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:50.724 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:50.719+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:51.110 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:51.106+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:51.179 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:51.174+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:51.331 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:51.326+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:51.365 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:51.360+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:51.481 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:51.476+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:51.650 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:51.645+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:51.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:51.760+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:52.149 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:52.144+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:52.156 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:52.151+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:52.296 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:52.291+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:52.321 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:52.316+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:52.435 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:52.430+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:52.657 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:52.652+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:52.777 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:52.773+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:53.121 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:53.116+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:53.168 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:53.163+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:53.302 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:53.297+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:53.364 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:53.359+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:53.401 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:53.396+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:53.705 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:53.700+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:53.760 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:53.755+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:54.133 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:54.128+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:54.166 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:54.161+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:54.265 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:54.257+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:54.398 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:54.393+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:54.405 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:54.401+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:54.749 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:54.744+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:54.821 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:54.815+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:55.145 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:55.139+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:55.175 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:55.169+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:55.301 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:55.296+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:55.368 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:55.362+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:55.369 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:55.364+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:55.771 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:55.765+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:55.871 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:55.865+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:56.128 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:56.123+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:56.220 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:56.215+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:56.344 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:56.338+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:56.358 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:56.352+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:56.379 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:56.374+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:56.806 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:56.800+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:56.839 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:56.835+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:57.140 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:57.134+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:57.203 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:57.197+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:57.332 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:57.327+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:57.361 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:57.356+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:57.391 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:57.385+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:57.791 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:57.786+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:57.855 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:57.849+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:58.142 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:58.137+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:58.224 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:58.218+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:58.328 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:58.323+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:58.337 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:58.332+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:58.348 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:58.342+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:58.810 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:58.806+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:58.903 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:58.897+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T16:59:59.130 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T16:59:59.124+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T16:59:59.248 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T16:59:59.242+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T16:59:59.330 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T16:59:59.325+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T16:59:59.347 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T16:59:59.342+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T16:59:59.393 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T16:59:59.387+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T16:59:59.769 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T16:59:59.765+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T16:59:59.942 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T16:59:59.937+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:00.097 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:00.090+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:00.207 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:00.201+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:00.303 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:00.298+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:00.337 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:00.331+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:00.338 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:00.334+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:00.401 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:00.394+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:00.756 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:00.751+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:00.953 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:00.949+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:01.134 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:01.127+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:01.183 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:01.177+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:01.280 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:01.274+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:01.298 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:01.294+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:01.299 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:01.295+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:01.364 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:01.357+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:01.751 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:01.746+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:01.911 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:01.906+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:02.093 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:02.087+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:02.167 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:02.161+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:02.293 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:02.288+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:02.338 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:02.334+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:02.339 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:02.335+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:02.348 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:02.341+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:02.742 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:02.736+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:02.911 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:02.906+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:03.105 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:03.099+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:03.142 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:03.137+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:03.309 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:03.303+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:03.334 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:03.328+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:03.373 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:03.365+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:03.377 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:03.370+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:03.723 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:03.718+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:03.944 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:03.939+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:04.123 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:04.118+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:04.197 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:04.192+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:04.301 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:04.297+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:04.315 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:04.310+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:04.354 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:04.348+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:04.354 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:04.350+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:04.724 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:04.719+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:04.910 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:04.906+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:05.154 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:05.148+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:05.201 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:05.195+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:05.265 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:05.260+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:05.333 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:05.328+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:05.334 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:05.329+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:05.363 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:05.355+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:05.776 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:05.770+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:05.895 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:05.889+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:06.149 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:06.142+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:06.245 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:06.239+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:06.257 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:06.251+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:06.370 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:06.366+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:06.372 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:06.369+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:06.398 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:06.392+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:06.770 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:06.763+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:06.939 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:06.933+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:07.118 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:07.111+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:07.245 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:07.239+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:07.245 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:07.240+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:07.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:07.348+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:07.354 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:07.349+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:07.431 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:07.424+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:07.817 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:07.810+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:07.980 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:07.974+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:08.116 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:08.110+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:08.244 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:08.237+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:08.288 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:08.282+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:08.334 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:08.329+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:08.335 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:08.331+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:08.415 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:08.409+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:08.773 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:08.769+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:09.003 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:08.997+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:09.089 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:09.083+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:09.255 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:09.249+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:09.275 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:09.270+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:09.316 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:09.311+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:09.317 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:09.313+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:09.428 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:09.423+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:09.728 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:09.723+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:10.022 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:10.018+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:10.023 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:00:10.015+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 43sec e405 v405) 2019-11-25T17:00:10.084 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:10.079+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:10.132 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:45:14.715 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.163 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:45:14.746 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.180 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:45:14.786 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.218 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:45:14.824 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.220 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:45:14.826 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.226 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:45:14.831 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.251 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:10.246+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:10.262 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:10.258+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:10.294 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:45:14.877 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.312 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:10.308+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:10.313 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:10.310+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:10.350 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:45:14.956 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.354 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:45:14.960 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:00:10.444 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:10.438+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:10.701 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:10.697+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:10.999 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:10.994+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:11.074 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:11.069+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:11.286 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:11.280+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:11.300 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:11.294+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:11.306 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:11.298+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:11.309 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:11.304+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:11.462 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:11.456+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:11.691 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:11.687+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:12.017 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:12.012+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:12.127 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:12.122+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:12.272 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:12.267+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:12.275 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:12.270+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:12.332 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:12.327+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:12.335 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:12.331+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:12.441 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:12.435+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:12.692 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:12.688+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:13.035 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:13.029+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:13.187 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:13.181+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:13.234 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:13.228+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:13.298 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:13.292+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:13.303 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:13.297+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:13.306 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:13.302+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:13.401 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:13.396+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:13.726 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:13.720+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:14.081 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:14.076+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:14.197 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:14.191+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:14.244 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:14.239+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:14.287 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:14.282+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:14.350 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:14.344+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:14.350 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:14.346+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:14.456 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:14.450+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:14.721 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:14.717+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:15.041 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:00:15.034+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 43sec e405 v405) 2019-11-25T17:00:15.093 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:15.089+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:15.198 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:15.193+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:15.286 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:15.282+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:15.325 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:15.319+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:15.381 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:15.375+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:15.381 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:15.377+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:15.490 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:15.485+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:15.705 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:15.699+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:16.127 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:16.122+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:16.231 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:16.225+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:16.315 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:16.310+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:16.367 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:16.361+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:16.394 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:16.389+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:16.394 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:16.390+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:16.510 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:16.504+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:16.747 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:16.742+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:17.087 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:17.081+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:17.238 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:17.232+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:17.328 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:17.321+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:17.354 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:17.348+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:17.394 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:17.386+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:17.398 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:17.388+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:17.517 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:17.511+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:17.769 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:17.763+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:18.084 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:18.079+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:18.249 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:18.241+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:18.366 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:18.361+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:18.375 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:18.370+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:18.435 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:18.429+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:18.435 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:18.431+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:18.569 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:18.564+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:18.811 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:18.806+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:19.057 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:19.052+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:19.272 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:19.267+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:19.384 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:00:19.379+0000 16949700 -1 osd.0 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.521468+0000 front 2019-11-25T16:58:56.075280+0000 (oldest deadline 2019-11-25T16:59:35.045419+0000) 2019-11-25T17:00:19.393 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:19.387+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:19.440 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:19.435+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:19.443 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:19.439+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:19.603 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:00:19.598+0000 16949700 -1 osd.5 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.405526+0000 front 2019-11-25T16:58:52.530325+0000 (oldest deadline 2019-11-25T16:59:34.708906+0000) 2019-11-25T17:00:19.865 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:00:19.859+0000 16949700 -1 osd.2 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.074865+0000 front 2019-11-25T16:58:54.687491+0000 (oldest deadline 2019-11-25T16:59:34.860636+0000) 2019-11-25T17:00:20.113 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:00:20.107+0000 16949700 -1 osd.1 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:55.417016+0000 front 2019-11-25T16:58:56.131973+0000 (oldest deadline 2019-11-25T16:59:39.120568+0000) 2019-11-25T17:00:20.138 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:00:20.125+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 43sec e405 v405) 2019-11-25T17:00:20.302 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:00:20.297+0000 16949700 -1 osd.7 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:54.769080+0000 front 2019-11-25T16:58:54.738174+0000 (oldest deadline 2019-11-25T16:59:36.672308+0000) 2019-11-25T17:00:20.409 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:20.403+0000 16949700 -1 osd.3 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:52.492990+0000 front 2019-11-25T16:58:52.507187+0000 (oldest deadline 2019-11-25T16:59:36.344807+0000) 2019-11-25T17:00:20.409 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:20.404+0000 16949700 -1 osd.3 405 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:20.435 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:00:20.430+0000 16949700 -1 osd.6 405 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T16:58:56.086362+0000 front 2019-11-25T16:58:56.075218+0000 (oldest deadline 2019-11-25T16:59:38.570451+0000) 2019-11-25T17:00:21.424 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:21.419+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:22.431 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:22.425+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:23.456 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:23.450+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:24.425 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:24.419+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:25.455 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:25.450+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:26.420 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:26.414+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:27.401 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:27.396+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:28.427 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:28.422+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:29.400 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:29.396+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:30.381 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:30.377+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:31.378 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:31.372+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:32.403 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:32.398+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:33.441 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:33.437+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:34.447 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:34.443+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:35.468 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:35.464+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:36.424 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:36.420+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:37.391 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:37.387+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:38.371 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:38.367+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:39.338 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:39.334+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:40.340 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:40.336+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:00:41.341 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:00:41.337+0000 16949700 -1 osd.3 407 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:120818 2.34s0 2:2c3e5454:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.657_%e0%a4%80%e8%94%88%e1%a0%82%e5%98%87%e2%98%83%ef%84%8f%e1%a0%83%e2%98%83%ef%84%92%e3%9c%85%e0%a4%82%e5%98%85%eb%8c%8d%ef%84%91%eb%8c%8e:head [cmpxattr user.rgw.idtag (48) op 1 mode 1 in=62b,call rgw.obj_remove in=40b] snapc 0=[] ondisk+write+known_if_redirected e405) 2019-11-25T17:01:17.935 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:17.929+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:17.953 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:17.948+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:18.689 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:18.684+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:18.947 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:18.941+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:18.960 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:18.947+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:19.667 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:19.660+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:19.671 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:19.665+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:19.907 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:19.901+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:19.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:19.914+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:19.921 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:19.915+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:20.429 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:20.424+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:20.662 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:20.656+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:20.678 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:20.672+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:20.686 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:20.680+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:20.906 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:20.900+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:20.946 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:20.940+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:20.963 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:20.957+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:21.427 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:21.423+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:21.619 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:21.612+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:21.661 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:21.655+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:21.698 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:21.694+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:21.927 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:21.920+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:21.933 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:21.929+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:21.999 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:21.993+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:22.428 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:22.424+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:22.593 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:22.589+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:22.621 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:22.617+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:22.715 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:22.711+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:22.929 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:22.924+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:22.975 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:22.970+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:23.019 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:23.014+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:23.417 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:23.413+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:23.610 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:23.605+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:23.639 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:23.634+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:23.733 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:23.728+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:23.909 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:23.904+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:23.985 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:23.975+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:24.046 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:24.041+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:24.431 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:24.428+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:24.584 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:24.579+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:24.661 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:24.657+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:24.725 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:24.721+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:24.865 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:24.861+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:25.025 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:25.020+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:25.104 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:25.100+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:25.391 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:25.387+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:25.631 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:25.626+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:25.684 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:25.680+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:25.699 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:25.694+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:25.868 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:25.863+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:26.006 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:26.001+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:26.132 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:26.127+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:26.377 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:26.373+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:26.652 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:26.647+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:26.663 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:26.659+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:26.720 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:26.716+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:26.847 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:26.841+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:27.048 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:27.042+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:27.164 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:27.159+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:27.361 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:27.356+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:27.362 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:27.358+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:27.680 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:27.676+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:27.702 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:27.696+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:27.738 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:27.733+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:27.831 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:27.826+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:28.020 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:28.016+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:28.357 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:28.352+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:28.359 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:28.355+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:28.376 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:28.358+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:28.681 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:28.672+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:28.706 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:28.702+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:28.723 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:28.719+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:28.842 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:28.838+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:29.018 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:29.014+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:29.348 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:29.343+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:29.351 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:29.347+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:29.365 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:29.360+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:29.644 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:29.639+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:29.696 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:29.691+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:29.756 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:29.752+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:29.885 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:29.879+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:30.038 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:30.033+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:30.387 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:30.382+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:30.399 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:30.396+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:30.401 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:30.398+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:30.669 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:30.664+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:30.741 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:30.737+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:30.807 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:30.803+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:30.846 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:30.841+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:31.091 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:31.086+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:31.373 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:31.369+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:31.374 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:31.370+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:31.416 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:31.412+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:31.647 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:31.642+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:31.721 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:31.716+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:31.848 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:31.843+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:31.886 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:31.881+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:32.142 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:32.137+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:32.407 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:32.403+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:32.407 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:32.404+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:32.419 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:32.408+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:32.609 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:32.603+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:32.720 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:32.715+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:32.868 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:32.863+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:32.891 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:32.886+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:33.126 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:33.121+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:33.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:33.379+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:33.386 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:33.381+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:33.427 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:33.422+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:33.616 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:33.612+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:33.832 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:33.827+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:33.885 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:33.880+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:33.912 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:33.908+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:34.168 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:34.162+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:34.391 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:34.385+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:34.410 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:34.404+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:34.411 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:34.405+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:34.605 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:34.601+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:34.870 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:34.864+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:34.876 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:34.871+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:34.931 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:34.926+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:35.154 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:35.149+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:35.387 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:35.380+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:35.423 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:35.418+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:35.425 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:35.420+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:35.613 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:35.610+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:35.897 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:35.892+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:35.906 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:35.900+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:35.942 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:35.937+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:36.128 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:36.123+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:36.352 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:36.346+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:36.475 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:36.470+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:36.476 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:36.471+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:36.619 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:36.613+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:36.924 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:36.917+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:36.924 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:36.918+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:36.966 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:36.960+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:37.160 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:37.156+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:37.327 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:37.323+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:37.505 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:37.499+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:37.505 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:37.501+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:37.580 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:37.576+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:37.902 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:37.899+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:37.906 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:37.901+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:37.965 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:37.960+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:38.124 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:38.120+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:38.339 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:38.333+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:38.488 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:38.481+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:38.489 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:38.483+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:38.580 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:38.576+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:38.932 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:38.927+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:38.957 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:38.952+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:38.988 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:38.983+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:39.150 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:39.145+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:39.368 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:39.360+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:39.512 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:39.508+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:39.514 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:39.509+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:39.589 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:39.584+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:39.903 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:39.899+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:39.918 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:39.913+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:40.013 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:40.007+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:40.154 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:40.149+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:40.457 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:40.452+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:40.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:40.558+0000 16949700 -1 osd.4 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:00:47.105927+0000 (oldest deadline 2019-11-25T17:01:27.105927+0000) 2019-11-25T17:01:40.565 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:40.560+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:40.602 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:40.597+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:40.885 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:40.880+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:40.961 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:40.956+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:40.989 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:40.984+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:41.169 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:41.163+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:41.424 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:41.418+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:41.560 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:41.555+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:41.605 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:41.599+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:41.868 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:41.862+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:41.965 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:41.960+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:42.009 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:42.003+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:42.185 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:42.180+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:42.386 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:42.380+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:42.600 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:42.595+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:42.635 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:42.630+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:42.857 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:42.852+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:42.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:42.940+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:42.992 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:42.986+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:43.217 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:43.212+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:43.413 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:43.407+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:43.619 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:43.614+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:43.656 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:43.646+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:43.885 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:43.880+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:43.936 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:43.932+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:43.964 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:43.958+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:44.194 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:44.188+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:44.539 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:44.508+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:44.607 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:44.603+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:44.629 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:44.624+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:44.900 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:44.896+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:44.915 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:44.910+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:45.004 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:44.999+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:45.221 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:45.216+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:45.575 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:45.570+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:45.582 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:45.575+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:45.652 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:45.647+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:45.896 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:45.891+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:45.954 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:45.949+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:46.025 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:46.019+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:46.240 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:46.235+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:46.579 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:46.574+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:46.581 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:46.576+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:46.652 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:46.647+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:46.884 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:46.880+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:46.961 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:46.956+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:47.069 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:47.063+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:47.200 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:47.195+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:47.606 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:47.601+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:47.628 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:47.623+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:47.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:47.660+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:47.901 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:47.896+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:47.998 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:47.993+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:48.044 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:48.038+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:48.186 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:48.181+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:48.638 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:48.633+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:48.706 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:48.701+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:48.731 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:48.648+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:48.946 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:48.941+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:49.006 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:49.000+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:49.034 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:49.029+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:49.159 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:49.153+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:49.673 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:49.668+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:49.690 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:49.685+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:49.733 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:49.728+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:49.967 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:49.963+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:50.023 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:50.017+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:50.087 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:50.082+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:50.211 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:50.206+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:50.409 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:01:50.400+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 43sec e411 v411) 2019-11-25T17:01:50.677 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:50.673+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:50.698 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:50.692+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:50.728 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:50.722+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:51.017 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:51.013+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:51.052 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:51.047+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:51.068 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:51.063+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:51.202 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:51.198+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:51.666 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:51.661+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:51.717 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:51.712+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:51.765 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:51.759+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:52.015 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:52.003+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:52.021 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:52.016+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:52.045 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:52.039+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:52.162 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:52.158+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:52.624 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:52.619+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:52.716 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:52.711+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:52.743 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:52.738+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:52.997 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:52.992+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:53.018 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:53.013+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:53.028 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:53.022+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:53.138 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:53.134+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:53.597 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:53.592+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:53.691 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:53.686+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:53.699 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:53.694+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:53.978 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:53.972+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:54.019 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:54.013+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:54.046 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:54.040+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:54.153 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:54.149+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:54.609 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:54.604+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:54.688 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:54.683+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:54.731 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:54.725+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:54.943 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:54.938+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:55.030 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:55.025+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:55.054 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:55.048+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:55.144 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:55.139+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:55.416 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:01:55.410+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 43sec e411 v411) 2019-11-25T17:01:55.646 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:55.641+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:55.664 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:55.659+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:55.742 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:55.735+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:55.968 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:55.962+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:56.025 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:56.020+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:56.095 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:56.090+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:56.161 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:56.155+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:56.634 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:56.629+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:56.698 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:56.690+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:56.761 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:56.725+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:56.969 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:56.964+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:57.011 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:57.007+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:57.112 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:57.107+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:57.187 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:57.183+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:57.610 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:57.605+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:57.654 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:57.648+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:57.714 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:57.709+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:57.993 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:57.987+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:58.021 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:58.016+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:58.133 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:58.128+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:58.191 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:58.185+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:58.628 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:58.620+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:58.650 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:58.644+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:58.725 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:58.721+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:58.990 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:58.984+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:01:59.031 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:01:59.026+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:01:59.137 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:01:59.132+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:01:59.204 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:01:59.199+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:01:59.653 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:01:59.647+0000 16949700 -1 osd.5 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:32.050878+0000 front 2019-11-25T17:00:32.035052+0000 (oldest deadline 2019-11-25T17:01:17.137544+0000) 2019-11-25T17:01:59.666 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:01:59.662+0000 16949700 -1 osd.0 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.730095+0000 front 2019-11-25T17:00:35.859759+0000 (oldest deadline 2019-11-25T17:01:19.652359+0000) 2019-11-25T17:01:59.752 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:01:59.747+0000 16949700 -1 osd.4 411 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:01:59.981 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:01:59.975+0000 16949700 -1 osd.7 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:36.327850+0000 front 2019-11-25T17:00:35.253808+0000 (oldest deadline 2019-11-25T17:01:17.709413+0000) 2019-11-25T17:02:00.047 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:02:00.042+0000 16949700 -1 osd.1 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.359622+0000 front 2019-11-25T17:00:33.638091+0000 (oldest deadline 2019-11-25T17:01:19.485459+0000) 2019-11-25T17:02:00.117 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:02:00.113+0000 16949700 -1 osd.3 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:35.380968+0000 front 2019-11-25T17:00:35.380452+0000 (oldest deadline 2019-11-25T17:01:17.840429+0000) 2019-11-25T17:02:00.250 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:02:00.245+0000 16949700 -1 osd.2 411 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:00:37.636196+0000 front 2019-11-25T17:00:37.637139+0000 (oldest deadline 2019-11-25T17:01:19.866473+0000) 2019-11-25T17:02:00.474 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:02:00.467+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 43sec e411 v411) 2019-11-25T17:02:00.972 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:02:00.963+0000 16949700 -1 osd.4 412 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e409) 2019-11-25T17:02:09.399 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:47:13.982 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:09.421 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:47:14.027 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:09.422 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:47:14.004 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:09.422 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:47:14.027 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:09.427 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:47:14.033 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:09.429 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:47:14.035 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:09.618 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:47:14.201 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:09.654 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:47:14.259 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:09.663 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:47:14.269 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:02:28.786 INFO:teuthology.orchestra.run.smithi153.stderr:testBadHeaders (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:02:28.835 INFO:teuthology.orchestra.run.smithi153.stderr:testBlankMetadataName (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:02:35.928 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:02:35.908+0000 3aa7b700 -1 osd.6 416 build_incremental_map_msg missing incremental map 416 2019-11-25T17:02:36.131 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:02:36.118+0000 3a27a700 -1 osd.6 416 build_incremental_map_msg missing incremental map 416 2019-11-25T17:02:36.216 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:02:36.168+0000 3a27a700 -1 osd.6 416 build_incremental_map_msg missing incremental map 416 2019-11-25T17:02:36.255 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:02:36.211+0000 3a27a700 -1 osd.6 416 build_incremental_map_msg missing incremental map 416 2019-11-25T17:02:36.315 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:02:36.263+0000 38a77700 -1 osd.6 416 build_incremental_map_msg missing incremental map 416 2019-11-25T17:02:36.431 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:02:36.315+0000 37274700 -1 osd.6 416 build_incremental_map_msg missing incremental map 416 2019-11-25T17:02:36.721 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:02:36.556+0000 37274700 -1 osd.6 416 build_incremental_map_msg missing incremental map 416 2019-11-25T17:03:36.616 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:36.610+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:37.485 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:37.480+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:37.630 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:37.623+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:37.961 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:37.955+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:38.177 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:38.172+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:38.483 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:38.477+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:38.597 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:38.589+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:38.742 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:38.736+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:38.923 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:38.917+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:39.153 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:39.147+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:39.484 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:39.480+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:39.587 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:39.582+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:39.708 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:39.703+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:39.970 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:39.965+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:40.057 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:40.053+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:40.109 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:40.103+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:40.451 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:40.446+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:40.586 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:40.581+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:40.675 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:40.668+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:40.928 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:40.923+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:41.022 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:41.015+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:41.126 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:41.121+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:41.504 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:41.499+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:41.638 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:41.633+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:41.658 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:41.652+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:41.970 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:41.964+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:41.993 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:41.988+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:42.131 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:42.124+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:42.371 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:42.365+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:42.524 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:42.520+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:42.602 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:42.597+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:42.698 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:42.692+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:42.996 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:42.991+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:43.003 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:42.999+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:43.117 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:43.112+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:43.392 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:43.387+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:43.497 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:43.491+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:43.614 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:43.609+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:43.687 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:43.681+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:43.962 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:43.957+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:44.010 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:44.005+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:44.094 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:44.089+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:44.378 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:44.372+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:44.538 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:44.533+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:44.616 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:44.611+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:44.715 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:44.708+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:44.956 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:44.951+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:45.031 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:45.025+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:45.071 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:45.066+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:45.411 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:45.405+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:45.507 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:45.502+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:45.632 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:45.626+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:45.743 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:45.738+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:45.970 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:45.966+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:46.076 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:46.070+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:46.108 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:46.103+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:46.425 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:46.419+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:46.560 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:46.555+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:46.687 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:46.681+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:46.745 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:46.738+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:47.008 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:47.003+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:47.043 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:47.038+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:47.074 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:47.070+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:47.463 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:47.456+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:47.590 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:47.584+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:47.740 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:47.732+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:47.740 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:47.733+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:48.060 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:48.055+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:48.094 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:48.089+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:48.112 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:48.107+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:48.453 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:48.447+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:48.633 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:48.628+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:48.705 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:48.699+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:48.791 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:48.786+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:49.033 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:49.027+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:49.124 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:49.117+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:49.127 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:49.122+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:49.498 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:49.492+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:49.593 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:49.588+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:49.701 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:49.695+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:49.781 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:49.775+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:50.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:50.049+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:50.104 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:50.099+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:50.162 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:50.156+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:50.527 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:50.521+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:50.601 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:50.596+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:50.750 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:50.744+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:50.763 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:50.758+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:51.050 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:51.045+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:51.147 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:51.142+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:51.164 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:51.159+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:51.520 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:51.515+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:51.638 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:51.633+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:51.731 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:51.725+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:51.739 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:51.734+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:52.011 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:52.006+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:52.151 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:52.145+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:52.212 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:52.206+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:52.532 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:52.526+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:52.597 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:52.592+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:52.705 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:52.698+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:52.733 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:52.726+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:52.972 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:52.967+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:53.179 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:53.173+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:53.193 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:53.188+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:53.569 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:53.564+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:53.594 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:53.589+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:53.726 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:53.720+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:53.743 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:53.738+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:53.945 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:53.940+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:54.192 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:54.186+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:54.230 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:54.224+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:54.600 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:54.595+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:54.626 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:54.620+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:54.740 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:54.735+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:54.760 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:54.755+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:54.993 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:54.988+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:55.180 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:55.173+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:55.233 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:55.227+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:55.566 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:55.560+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:55.644 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:55.638+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:55.724 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:55.720+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:55.747 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:55.741+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:56.025 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:56.018+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:56.192 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:56.186+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:56.217 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:56.211+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:56.583 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:56.578+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:56.671 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:56.664+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:56.713 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:56.707+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:56.749 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:56.745+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:57.001 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:56.994+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:57.236 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:57.231+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:57.266 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:57.260+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:57.630 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:57.625+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:57.706 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:57.700+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:57.713 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:57.708+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:57.725 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:57.719+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:57.977 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:57.970+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:58.249 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:58.243+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:58.301 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:58.286+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:58.665 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:58.661+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:58.698 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:58.692+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:58.728 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:58.723+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:03:58.750 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:58.746+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:58.995 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:58.988+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:03:59.233 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:03:59.226+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:03:59.336 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:03:59.331+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:03:59.663 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:03:59.656+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:03:59.680 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:03:59.675+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:03:59.711 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:03:59.706+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:03:59.724 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:03:59.718+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:00.002 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:03:59.996+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:00.234 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:00.228+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:00.327 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:00.321+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:00.621 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:00.615+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:00.703 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:00.698+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:00.738 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:00.733+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:00.770 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:04:00.765+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:01.050 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:01.043+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:01.269 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:01.263+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:01.342 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:01.336+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:01.662 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:01.656+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:01.708 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:01.703+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:01.774 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:01.769+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:01.811 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:04:01.806+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:02.081 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:02.076+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:02.265 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:02.260+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:02.332 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:02.327+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:02.618 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:02.612+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:02.740 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:02.731+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:02.789 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:02.783+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:02.829 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:04:02.823+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:03.047 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:03.041+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:03.295 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:03.290+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:03.347 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:03.342+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:03.613 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:03.609+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:03.710 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:03.706+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:03.817 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:03.811+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:03.870 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:04:03.866+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:04.006 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:04.001+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:04.252 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:04.247+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:04.393 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:04.382+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:04.620 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:04.616+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:04.741 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:04.737+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:04.868 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:04.864+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:04.871 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:04:04.866+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:05.055 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:05.050+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:05.296 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:05.292+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:05.423 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:05.417+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:05.598 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:05.592+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:05.776 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:05.772+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:05.845 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:04:05.841+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:05.902 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:05.898+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:06.062 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:06.057+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:06.273 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:06.267+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:06.477 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:06.471+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:06.642 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:06.637+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:06.741 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:06.734+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:06.808 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:04:06.802+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:06.899 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:06.892+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:07.026 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:07.017+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:07.302 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:07.297+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:07.509 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:07.504+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:07.688 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:07.683+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:07.713 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:07.707+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:07.780 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:04:07.774+0000 16949700 -1 osd.5 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:55.788645+0000 front 2019-11-25T17:02:56.554543+0000 (oldest deadline 2019-11-25T17:03:37.984186+0000) 2019-11-25T17:04:07.944 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:07.938+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:08.003 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:07.998+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:08.340 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:08.334+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:08.464 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:08.459+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:08.687 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:08.677+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:08.710 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:08.704+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:08.985 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:08.979+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:09.012 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:09.006+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:09.364 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:09.360+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:09.436 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:09.430+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:09.714 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:09.710+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:09.758 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:09.752+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:10.004 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:09.998+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:10.048 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:10.042+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:10.353 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:10.347+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:10.482 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:10.477+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:10.737 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:10.732+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:10.793 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:10.787+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:10.895 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:04:10.889+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 45sec e417 v417) 2019-11-25T17:04:11.019 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:11.014+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:11.057 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:11.052+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:11.330 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:11.324+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:11.526 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:11.521+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:11.788 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:11.782+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:11.832 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:11.825+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:12.033 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:12.026+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:12.058 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:12.052+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:12.365 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:12.359+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:12.559 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:12.554+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:12.766 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:12.761+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:12.789 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:12.783+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:13.063 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:13.058+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:13.110 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:13.105+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:13.330 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:13.324+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:13.572 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:13.567+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:13.755 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:13.750+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:13.791 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:13.785+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:14.061 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:14.055+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:14.077 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:14.071+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:14.322 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:14.318+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:14.579 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:14.574+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:14.772 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:14.767+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:14.780 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:14.775+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:15.063 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:15.057+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:15.093 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:15.087+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:15.366 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:15.361+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:15.586 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:15.580+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:15.731 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:15.726+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:15.782 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:15.776+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:15.909 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:04:15.905+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 45sec e417 v417) 2019-11-25T17:04:16.065 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:16.059+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:16.124 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:16.118+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:16.412 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:16.408+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:16.557 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:16.551+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:16.747 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:16.741+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:16.764 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:16.759+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:17.026 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:17.021+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:17.122 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:17.117+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:17.403 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:17.398+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:17.582 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:17.576+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:17.711 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:17.704+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:17.767 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:17.762+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:18.013 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:18.007+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:18.130 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:18.125+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:18.416 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:18.404+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:18.583 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:18.577+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:18.744 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:18.738+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:18.807 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:18.802+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:19.017 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:19.011+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:19.131 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:19.125+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:19.428 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:19.423+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:19.579 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:19.573+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:19.794 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:19.788+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:19.814 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:19.809+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:19.994 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:19.989+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:20.131 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:04:20.126+0000 16949700 -1 osd.0 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.340096+0000 front 2019-11-25T17:02:57.113683+0000 (oldest deadline 2019-11-25T17:03:37.902802+0000) 2019-11-25T17:04:20.391 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:04:20.385+0000 16949700 -1 osd.1 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.931348+0000 front 2019-11-25T17:02:58.203137+0000 (oldest deadline 2019-11-25T17:03:39.043797+0000) 2019-11-25T17:04:20.567 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:04:20.562+0000 16949700 -1 osd.2 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:57.926944+0000 front 2019-11-25T17:02:57.740198+0000 (oldest deadline 2019-11-25T17:03:37.700718+0000) 2019-11-25T17:04:20.792 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:04:20.786+0000 16949700 -1 osd.6 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:58.370552+0000 front 2019-11-25T17:02:57.430203+0000 (oldest deadline 2019-11-25T17:03:41.856310+0000) 2019-11-25T17:04:20.800 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:04:20.795+0000 16949700 -1 osd.3 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:56.486899+0000 front 2019-11-25T17:02:57.994004+0000 (oldest deadline 2019-11-25T17:03:36.817709+0000) 2019-11-25T17:04:20.968 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:04:20.963+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 45sec e417 v417) 2019-11-25T17:04:21.037 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:04:21.032+0000 16949700 -1 osd.7 417 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:02:51.886950+0000 front 2019-11-25T17:02:57.945348+0000 (oldest deadline 2019-11-25T17:03:36.575635+0000) 2019-11-25T17:04:41.214 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:49:45.796 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:04:41.238 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:49:45.820 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:04:41.246 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:49:45.852 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:04:41.252 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:49:45.857 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:04:41.294 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:49:45.900 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:04:41.305 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:49:45.910 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:04:41.380 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:49:45.963 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:04:41.440 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:49:46.046 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:04:41.446 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:49:46.051 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:05:17.727 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:17.722+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:17.821 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:17.816+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:18.281 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:18.277+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:18.683 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:18.678+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:18.788 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:18.782+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:19.329 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:19.324+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:19.426 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:19.421+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:19.642 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:19.636+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:19.830 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:19.823+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:20.312 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:20.308+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:20.324 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:20.319+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:20.383 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:20.378+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:20.629 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:20.623+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:20.798 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:20.793+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:21.232 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:05:21.227+0000 16949700 -1 osd.5 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:35.111426+0000 front 2019-11-25T17:04:35.116044+0000 (oldest deadline 2019-11-25T17:05:20.740936+0000) 2019-11-25T17:05:21.304 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:21.299+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:21.327 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:21.321+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:21.386 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:21.381+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:21.666 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:21.661+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:21.714 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:21.710+0000 16949700 -1 osd.4 423 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:21.796 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:21.790+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:22.203 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:05:22.197+0000 16949700 -1 osd.5 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:35.111426+0000 front 2019-11-25T17:04:35.116044+0000 (oldest deadline 2019-11-25T17:05:20.740936+0000) 2019-11-25T17:05:22.293 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:22.287+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:22.303 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:22.298+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:22.437 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:22.432+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:22.695 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:22.690+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:22.768 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:22.763+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:22.787 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:22.782+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:23.261 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:23.255+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:23.269 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:23.264+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:23.453 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:23.447+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:23.656 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:23.651+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:23.734 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:23.729+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:23.835 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:23.831+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:24.231 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:24.224+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:24.294 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:24.289+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:24.457 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:24.451+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:24.660 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:24.656+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:24.733 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:24.727+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:24.807 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:24.803+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:25.274 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:25.268+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:25.275 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:25.270+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:25.413 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:25.408+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:25.658 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:25.653+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:25.762 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:25.757+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:25.828 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:25.823+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:26.269 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:26.264+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:26.309 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:26.303+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:26.469 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:26.463+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:26.673 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:26.668+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:26.726 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:26.721+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:26.794 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:26.789+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:27.311 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:27.307+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:27.354 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:27.348+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:27.421 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:27.418+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:27.677 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:27.673+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:27.735 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:27.728+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:27.830 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:27.826+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:28.321 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:28.317+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:28.393 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:28.388+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:28.398 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:28.394+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:28.651 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:28.647+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:28.719 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:28.712+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:28.719 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:28.714+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:28.797 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:28.790+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:29.352 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:29.347+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:29.363 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:29.358+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:29.370 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:29.364+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:29.701 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:29.696+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:29.771 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:29.765+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:29.773 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:29.767+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:29.835 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:29.830+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:30.336 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:30.330+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:30.355 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:30.350+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:30.375 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:30.370+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:30.712 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:30.708+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:30.739 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:30.733+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:30.742 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:30.737+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:30.828 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:30.823+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:31.341 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:31.337+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:31.353 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:31.349+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:31.364 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:31.359+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:31.677 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:31.672+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:31.729 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:31.724+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:31.731 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:31.727+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:31.797 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:31.792+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:32.348 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:32.342+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:32.351 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:32.346+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:32.394 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:32.389+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:32.660 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:32.655+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:32.731 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:32.725+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:32.732 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:32.727+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:32.845 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:32.840+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:33.368 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:33.362+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:33.378 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:33.373+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:33.399 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:33.394+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:33.631 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:33.626+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:33.784 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:33.777+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:33.785 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:33.779+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:33.868 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:33.863+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:34.326 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:34.320+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:34.412 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:34.408+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:34.419 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:34.415+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:34.626 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:34.622+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:34.786 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:34.782+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:34.789 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:34.785+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:34.858 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:34.852+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:35.319 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:35.313+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:35.434 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:35.429+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:35.456 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:35.452+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:35.592 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:35.587+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:35.821 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:35.817+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:35.824 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:35.821+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:35.835 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:35.830+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:36.289 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:36.285+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:36.460 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:36.455+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:36.507 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:36.502+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:36.582 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:36.578+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:36.792 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:36.787+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:36.793 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:36.788+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:36.869 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:36.864+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:37.301 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:37.296+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:37.429 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:37.425+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:37.475 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:37.470+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:37.579 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:37.574+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:37.832 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:37.827+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:37.832 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:37.828+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:37.921 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:37.915+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:38.270 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:38.266+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:38.426 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:38.419+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:38.454 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:38.449+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:38.592 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:38.587+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:38.885 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:38.881+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:38.888 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:38.884+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:38.941 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:38.937+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:39.279 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:39.274+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:39.444 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:39.439+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:39.455 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:39.450+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:39.629 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:39.624+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:39.923 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:39.916+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:39.924 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:39.919+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:39.979 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:39.975+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:40.312 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:40.308+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:40.402 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:40.397+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:40.469 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:40.463+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:40.598 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:40.593+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:40.943 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:40.938+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:40.963 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:40.958+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:40.963 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:40.959+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:41.271 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:41.266+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:41.436 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:41.430+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:41.452 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:41.446+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:41.564 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:41.558+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:41.995 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:41.989+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:42.009 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:42.003+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:42.010 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:42.005+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:42.251 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:42.247+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:42.419 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:42.412+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:42.472 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:42.465+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:42.531 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:42.525+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:43.042 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:43.036+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:43.042 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:43.038+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:43.045 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:43.039+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:43.236 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:43.231+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:43.410 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:43.404+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:43.502 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:43.496+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:43.512 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:43.507+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:44.075 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:44.069+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:44.075 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:44.071+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:44.105 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:44.094+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:44.256 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:44.252+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:44.379 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:44.372+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:44.479 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:44.473+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:44.553 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:44.548+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:45.061 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:45.055+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:45.062 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:45.057+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:45.082 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:45.076+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:45.253 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:45.248+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:45.373 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:45.365+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:45.480 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:45.474+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:45.515 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:45.510+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:46.046 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:46.040+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:46.050 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:46.046+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:46.052 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:46.049+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:46.253 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:46.249+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:46.377 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:46.371+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:46.496 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:46.490+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:46.522 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:46.516+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:47.014 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:47.007+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:47.102 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:47.097+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:47.103 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:47.099+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:47.244 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:47.239+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:47.332 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:47.326+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:47.496 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:47.487+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:47.540 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:47.534+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:48.053 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:48.047+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:48.114 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:48.109+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:48.118 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:48.113+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:48.268 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:48.263+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:48.364 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:48.360+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:48.487 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:48.482+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:48.576 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:48.571+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:49.102 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:49.096+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:49.139 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:49.134+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:49.142 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:49.138+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:49.313 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:49.307+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:49.386 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:49.382+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:49.461 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:49.457+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:49.576 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:49.570+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:50.097 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:50.091+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:50.140 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:50.134+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:50.141 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:50.135+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:50.305 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:50.299+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:50.408 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:50.403+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:50.429 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:50.424+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:50.536 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:50.532+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:51.133 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:51.128+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:51.144 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:51.138+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:51.145 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:51.140+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:51.295 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:51.289+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:51.401 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:51.396+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:51.432 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:51.427+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:51.495 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:51.489+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:51.497 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:05:51.491+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e423 v423) 2019-11-25T17:05:52.123 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:52.117+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:52.124 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:52.118+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:52.173 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:52.168+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:52.334 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:52.329+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:52.439 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:52.434+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:52.455 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:52.450+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:52.506 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:52.500+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:53.079 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:53.073+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:53.080 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:53.074+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:53.150 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:53.145+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:53.384 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:53.379+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:53.438 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:53.434+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:53.482 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:53.476+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:53.517 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:53.512+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:54.075 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:54.069+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:54.076 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:54.070+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:54.202 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:54.196+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:54.431 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:54.425+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:54.471 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:54.467+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:54.505 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:54.500+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:54.547 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:54.543+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:55.069 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:55.062+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:55.070 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:55.064+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:55.191 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:55.186+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:55.385 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:55.380+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:55.434 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:55.430+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:55.504 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:55.498+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:55.570 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:55.565+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:56.078 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:56.072+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:56.079 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:56.073+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:56.201 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:56.197+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:56.374 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:56.368+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:56.414 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:56.409+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:56.506 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:56.501+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:56.516 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:05:56.512+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e423 v423) 2019-11-25T17:05:56.531 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:56.525+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:57.063 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:57.058+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:57.064 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:57.058+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:57.247 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:57.242+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:57.412 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:57.407+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:57.426 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:57.421+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:57.509 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:57.505+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:57.524 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:57.519+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:58.026 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:58.021+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:58.030 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:58.025+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:58.272 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:58.267+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:58.417 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:58.411+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:58.459 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:58.454+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:58.516 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:58.511+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:58.551 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:58.541+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:05:59.070 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:59.063+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:05:59.071 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:05:59.065+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:05:59.240 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:05:59.235+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:05:59.437 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:05:59.432+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:05:59.485 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:05:59.479+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:05:59.485 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:05:59.480+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:05:59.561 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:05:59.556+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:06:00.044 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:00.038+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:06:00.046 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:00.041+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:06:00.272 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:06:00.267+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:06:00.444 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:06:00.439+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:06:00.470 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:06:00.464+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:06:00.488 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:06:00.482+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:06:00.604 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:06:00.600+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:06:01.057 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:01.050+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:06:01.057 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:01.052+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:06:01.297 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:06:01.291+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:06:01.441 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:06:01.436+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:06:01.449 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:06:01.445+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:06:01.481 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:06:01.476+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:06:01.540 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:06:01.531+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e423 v423) 2019-11-25T17:06:01.562 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:06:01.557+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:06:02.068 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:02.063+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:06:02.069 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:02.064+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:06:02.254 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:06:02.248+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:06:02.465 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:06:02.461+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:06:02.480 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:06:02.475+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:06:02.489 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:06:02.485+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:06:02.587 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:06:02.582+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:06:03.027 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:03.021+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:06:03.028 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:03.023+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:06:03.304 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:06:03.298+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:06:03.475 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:06:03.470+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:06:03.476 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:06:03.471+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:06:03.519 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:06:03.513+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:06:03.593 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:06:03.586+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:06:04.077 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:04.070+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:06:04.077 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:04.072+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:06:04.276 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:06:04.271+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:06:04.439 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:06:04.434+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:06:04.488 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:06:04.482+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:06:04.515 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:06:04.510+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:06:04.581 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:06:04.577+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:06:05.088 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:05.082+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:06:05.092 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:05.087+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:06:05.245 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:06:05.240+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:06:05.278 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:06:05.272+0000 16949700 -1 osd.5 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:35.111426+0000 front 2019-11-25T17:04:35.116044+0000 (oldest deadline 2019-11-25T17:06:05.116310+0000) 2019-11-25T17:06:05.462 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:06:05.457+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:06:05.470 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:06:05.465+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:06:05.545 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:06:05.536+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:06:05.611 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:06:05.607+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:06:06.078 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:06.071+0000 16949700 -1 osd.4 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:04:48.382674+0000 (oldest deadline 2019-11-25T17:05:28.382674+0000) 2019-11-25T17:06:06.079 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:06:06.073+0000 16949700 -1 osd.4 423 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e421) 2019-11-25T17:06:06.223 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:06:06.219+0000 16949700 -1 osd.0 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:31.944760+0000 front 2019-11-25T17:04:38.221774+0000 (oldest deadline 2019-11-25T17:05:17.346908+0000) 2019-11-25T17:06:06.294 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:06:06.287+0000 16949700 -1 osd.5 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:35.111426+0000 front 2019-11-25T17:04:35.116044+0000 (oldest deadline 2019-11-25T17:06:05.116310+0000) 2019-11-25T17:06:06.428 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:06:06.423+0000 16949700 -1 osd.2 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:38.011595+0000 front 2019-11-25T17:04:37.148228+0000 (oldest deadline 2019-11-25T17:05:17.682290+0000) 2019-11-25T17:06:06.455 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:06:06.450+0000 16949700 -1 osd.1 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.845047+0000 front 2019-11-25T17:04:34.855763+0000 (oldest deadline 2019-11-25T17:05:17.451267+0000) 2019-11-25T17:06:06.569 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:06:06.563+0000 16949700 -1 osd.3 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:34.817402+0000 front 2019-11-25T17:04:34.795665+0000 (oldest deadline 2019-11-25T17:05:18.674344+0000) 2019-11-25T17:06:06.595 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:06:06.590+0000 16949700 -1 osd.7 423 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:04:36.390093+0000 front 2019-11-25T17:04:36.263113+0000 (oldest deadline 2019-11-25T17:05:19.680110+0000) 2019-11-25T17:06:06.623 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:06:06.614+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e423 v423) 2019-11-25T17:06:07.304 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:51:11.886 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:07.311 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:51:11.894 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:07.323 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:51:11.929 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:07.336 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:51:11.935 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:07.336 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:51:11.938 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:07.344 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:51:11.945 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:07.424 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:51:12.006 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:07.517 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:51:12.123 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:07.570 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:51:12.174 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:06:37.444 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:06:37.440+0000 11ca9700 -1 osd.6 428 build_incremental_map_msg missing incremental map 428 2019-11-25T17:07:19.721 INFO:teuthology.orchestra.run.smithi153.stderr:testChunkedPut (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:07:38.218 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:38.213+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:39.228 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:39.223+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:39.265 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:39.259+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:39.402 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:39.397+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:39.565 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:39.559+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:39.644 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:39.638+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:40.205 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:40.200+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:40.219 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:40.214+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:40.442 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:40.437+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:40.577 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:40.571+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:40.627 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:40.622+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:41.180 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:41.175+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:41.248 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:41.242+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:41.459 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:41.455+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:41.575 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:41.567+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:41.598 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:41.593+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:42.198 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:42.192+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:42.235 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:42.230+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:42.252 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:42.247+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:42.456 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:42.451+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:42.571 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:42.566+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:42.623 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:42.617+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:43.234 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:43.229+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:43.242 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:43.237+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:43.285 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:43.278+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:43.433 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:43.426+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:43.505 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:43.500+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:43.587 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:43.581+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:43.599 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:43.593+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:44.250 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:44.244+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:44.262 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:44.256+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:44.273 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:44.267+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:44.413 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:44.406+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:44.550 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:44.544+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:44.562 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:44.557+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:44.583 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:44.577+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:45.243 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:45.237+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:45.256 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:45.251+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:45.283 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:45.278+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:45.452 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:45.446+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:45.525 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:45.520+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:45.539 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:45.535+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:45.570 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:45.565+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:46.215 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:46.210+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:46.279 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:46.274+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:46.317 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:46.312+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:46.445 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:46.439+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:46.486 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:46.480+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:46.519 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:46.515+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:46.570 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:46.565+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:47.240 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:47.233+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:47.273 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:47.267+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:47.301 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:47.296+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:47.454 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:47.448+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:47.456 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:47.450+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:47.545 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:47.539+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:47.593 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:47.589+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:48.251 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:48.246+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:48.287 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:48.281+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:48.313 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:48.309+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:48.452 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:48.445+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:48.464 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:48.458+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:48.576 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:48.570+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:48.579 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:48.573+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:49.272 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:49.267+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:49.281 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:49.271+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:49.289 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:49.284+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:49.445 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:49.440+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:49.471 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:49.465+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:49.569 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:49.563+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:49.592 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:49.587+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:50.257 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:50.251+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:50.271 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:50.265+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:50.333 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:50.326+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:50.425 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:50.420+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:50.452 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:50.446+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:50.563 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:50.557+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:50.643 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:50.638+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:51.248 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:51.242+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:51.304 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:51.298+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:51.360 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:51.353+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:51.383 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:51.377+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:51.438 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:51.432+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:51.535 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:51.530+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:51.680 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:51.674+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:52.262 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:52.257+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:52.300 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:52.294+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:52.377 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:52.370+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:52.394 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:52.389+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:52.404 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:52.399+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:52.547 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:52.542+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:52.680 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:52.675+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:53.255 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:53.249+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:53.301 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:53.296+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:53.380 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:53.374+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:53.418 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:53.412+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:53.421 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:53.415+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:53.552 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:53.547+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:53.706 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:53.701+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:54.265 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:54.260+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:54.344 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:54.337+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:54.387 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:54.381+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:54.437 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:54.426+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:54.486 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:54.443+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:54.594 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:54.589+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:54.674 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:54.669+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:55.293 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:55.287+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:55.329 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:55.324+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:55.369 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:55.363+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:55.425 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:55.419+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:55.480 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:55.475+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:55.587 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:55.581+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:55.703 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:55.698+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:56.266 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:56.260+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:56.378 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:56.371+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:56.400 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:56.393+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:56.402 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:56.396+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:56.528 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:56.521+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:56.633 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:56.628+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:56.718 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:56.712+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:57.308 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:57.302+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:57.341 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:57.335+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:57.427 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:57.419+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:57.428 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:57.422+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:57.575 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:57.570+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:57.654 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:57.650+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:57.725 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:57.718+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:58.343 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:58.337+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:58.347 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:58.341+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:58.432 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:58.427+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:58.476 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:58.469+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:58.591 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:58.585+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:58.646 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:58.641+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:58.701 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:58.696+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:07:59.343 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:07:59.337+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:07:59.380 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:07:59.375+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:07:59.395 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:07:59.389+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:07:59.457 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:07:59.451+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:07:59.626 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:07:59.620+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:07:59.628 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:07:59.623+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:07:59.705 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:07:59.700+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:00.337 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:00.332+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:00.357 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:00.352+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:00.397 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:00.391+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:00.475 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:00.469+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:00.599 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:00.593+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:00.658 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:00.653+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:00.740 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:00.731+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:01.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:01.348+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:01.369 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:01.363+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:01.445 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:01.439+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:01.501 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:01.496+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:01.566 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:01.560+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:01.694 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:01.688+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:01.781 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:01.776+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:02.337 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:02.332+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:02.409 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:02.405+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:02.437 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:02.432+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:02.476 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:02.469+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:02.585 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:02.580+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:02.681 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:02.676+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:02.779 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:02.773+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:03.333 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:03.329+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:03.397 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:03.390+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:03.433 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:03.427+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:03.465 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:03.459+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:03.601 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:03.596+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:03.697 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:03.691+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:03.771 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:03.765+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:04.316 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:04.309+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:04.361 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:04.355+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:04.461 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:04.456+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:04.496 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:04.491+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:04.556 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:04.551+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:04.748 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:04.742+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:04.818 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:04.813+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:05.325 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:05.320+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:05.389 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:05.383+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:05.451 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:05.446+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:05.461 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:05.455+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:05.534 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:05.529+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:05.776 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:05.769+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:05.780 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:05.774+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:06.359 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:06.354+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:06.383 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:06.378+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:06.424 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:06.419+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:06.483 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:06.478+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:06.569 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:06.563+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:06.742 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:06.736+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:06.773 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:06.768+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:07.384 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:07.379+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:07.406 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:07.401+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:07.437 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:07.432+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:07.487 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:07.481+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:07.537 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:07.531+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:07.735 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:07.730+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:07.813 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:07.807+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:08.428 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:08.423+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:08.440 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:08.435+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:08.471 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:08.467+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:08.496 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:08.490+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:08.549 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:08.544+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:08.720 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:08.715+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:08.776 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:08.771+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:09.415 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:09.411+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:09.419 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:09.414+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:09.505 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:09.500+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:09.510 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:09.503+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:09.513 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:09.507+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:09.758 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:09.753+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:09.812 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:09.807+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:10.395 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:10.389+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:10.403 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:10.397+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:10.471 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:10.465+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:10.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:10.469+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:10.548 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:10.542+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:10.779 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:10.776+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:10.828 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:10.822+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:11.416 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:11.412+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:11.445 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:11.440+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:11.466 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:11.462+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:11.513 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:11.507+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:11.524 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:11.518+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:11.783 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:11.778+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:11.843 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:11.838+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:11.980 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:08:11.975+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 46sec e428 v428) 2019-11-25T17:08:12.424 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:12.420+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:12.494 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:12.489+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:12.496 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:12.491+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:12.513 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:12.508+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:12.541 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:12.536+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:12.773 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:12.768+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:12.888 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:12.883+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:13.382 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:13.377+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:13.481 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:13.476+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:13.486 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:13.482+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:13.521 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:13.516+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:13.530 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:13.526+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:13.728 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:13.724+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:13.928 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:13.923+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:14.355 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:14.349+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:14.467 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:14.457+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:14.510 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:14.505+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:14.552 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:14.547+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:14.566 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:14.561+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:14.697 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:14.690+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:14.908 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:14.903+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:15.311 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:15.307+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:15.450 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:15.445+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:15.508 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:15.503+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:15.559 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:15.554+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:15.594 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:15.588+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:15.735 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:15.730+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:15.888 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:15.883+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:16.285 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:08:16.280+0000 16949700 -1 osd.6 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:57.772994+0000 front 2019-11-25T17:06:57.771232+0000 (oldest deadline 2019-11-25T17:07:38.764590+0000) 2019-11-25T17:08:16.485 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:08:16.480+0000 16949700 -1 osd.0 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:53.483642+0000 front 2019-11-25T17:06:53.441621+0000 (oldest deadline 2019-11-25T17:07:38.535640+0000) 2019-11-25T17:08:16.530 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:16.525+0000 16949700 -1 osd.2 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.844707+0000 front 2019-11-25T17:06:56.085661+0000 (oldest deadline 2019-11-25T17:07:41.528133+0000) 2019-11-25T17:08:16.592 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:08:16.588+0000 16949700 -1 osd.5 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.839053+0000 front 2019-11-25T17:06:55.551063+0000 (oldest deadline 2019-11-25T17:07:38.147078+0000) 2019-11-25T17:08:16.625 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:08:16.620+0000 16949700 -1 osd.3 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.155631+0000 front 2019-11-25T17:06:55.175721+0000 (oldest deadline 2019-11-25T17:07:38.668711+0000) 2019-11-25T17:08:16.783 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:08:16.779+0000 16949700 -1 osd.7 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:07:01.228787+0000 front 2019-11-25T17:06:57.772956+0000 (oldest deadline 2019-11-25T17:07:42.665961+0000) 2019-11-25T17:08:16.932 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:16.927+0000 16949700 -1 osd.1 428 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:06:55.300191+0000 front 2019-11-25T17:06:55.176653+0000 (oldest deadline 2019-11-25T17:07:38.718533+0000) 2019-11-25T17:08:17.063 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:08:17.058+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 46sec e428 v428) 2019-11-25T17:08:50.679 INFO:teuthology.orchestra.run.smithi153.stderr:testContentTypeGuessing (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:08:56.658 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:08:56.654+0000 39278700 -1 osd.2 432 build_incremental_map_msg missing incremental map 432 2019-11-25T17:08:56.786 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:56.759+0000 35270700 -1 osd.1 432 build_incremental_map_msg missing incremental map 432 2019-11-25T17:08:56.839 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:56.830+0000 37274700 -1 osd.1 432 build_incremental_map_msg missing incremental map 432 2019-11-25T17:08:57.054 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:08:57.037+0000 37274700 -1 osd.1 432 build_incremental_map_msg missing incremental map 432 2019-11-25T17:09:05.278 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:05.274+0000 38a77700 -1 osd.2 433 build_incremental_map_msg missing incremental map 432 2019-11-25T17:09:12.326 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:12.321+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:13.115 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:13.109+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:13.297 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:13.291+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:14.089 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:14.083+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:14.289 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:14.283+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:14.626 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:14.620+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:15.074 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:15.067+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:15.259 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:15.254+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:15.669 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:15.663+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:16.051 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:16.045+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:16.286 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:16.280+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:16.548 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:16.542+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:16.652 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:16.646+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:16.698 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:16.693+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:16.872 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:16.866+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:17.068 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:17.063+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:17.270 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:17.264+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:17.554 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:17.548+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:17.637 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:17.631+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:17.706 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:17.699+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:17.836 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:17.831+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:18.119 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:18.112+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:18.286 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:18.281+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:18.605 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:18.600+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:18.660 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:18.654+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:18.754 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:18.748+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:18.813 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:18.807+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:19.122 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:19.115+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:19.275 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:19.269+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:19.648 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:19.642+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:19.650 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:19.645+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:19.783 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:19.778+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:19.795 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:19.790+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:20.102 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:20.098+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:20.234 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:20.228+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:20.650 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:20.644+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:20.656 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:20.652+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:20.802 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:20.797+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:20.809 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:20.804+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:21.157 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:21.152+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:21.276 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:21.270+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:21.641 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:21.635+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:21.665 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:21.660+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:21.763 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:21.759+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:21.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:21.760+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:22.172 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:22.167+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:22.310 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:22.302+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:22.677 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:22.672+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:22.716 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:22.711+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:22.724 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:22.719+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:22.764 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:22.759+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:23.185 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:23.181+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:23.311 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:23.306+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:23.640 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:23.634+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:23.688 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:23.684+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:23.742 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:23.737+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:23.804 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:23.799+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:24.157 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:24.150+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:24.313 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:24.308+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:24.657 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:24.652+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:24.664 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:24.660+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:24.762 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:24.756+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:24.846 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:24.841+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:25.159 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:25.154+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:25.162 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:25.158+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:25.355 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:25.350+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:25.623 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:25.618+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:25.687 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:25.683+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:25.729 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:25.724+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:25.850 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:25.845+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:26.164 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:26.159+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:26.164 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:26.160+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:26.346 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:26.341+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:26.603 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:26.598+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:26.688 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:26.683+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:26.733 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:26.729+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:26.885 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:26.880+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:27.161 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:27.156+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:27.165 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:27.160+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:27.340 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:27.335+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:27.530 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:27.525+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:27.580 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:27.574+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:27.705 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:27.700+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:27.714 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:27.710+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:27.868 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:27.863+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:28.149 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:28.144+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:28.151 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:28.148+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:28.350 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:28.345+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:28.568 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:28.563+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:28.608 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:28.603+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:28.667 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:28.663+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:28.673 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:28.669+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:28.875 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:28.870+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:29.202 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:29.197+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:29.204 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:29.200+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:29.362 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:29.356+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:29.585 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:29.579+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:29.635 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:29.630+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:29.641 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:29.637+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:29.695 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:29.691+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:29.838 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:29.833+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:30.186 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:30.181+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:30.186 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:30.182+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:30.388 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:30.382+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:30.597 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:30.592+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:30.617 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:30.613+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:30.675 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:30.670+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:30.685 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:30.680+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:30.883 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:30.879+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:31.233 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:31.227+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:31.233 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:31.229+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:31.379 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:31.373+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:31.609 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:31.604+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:31.647 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:31.641+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:31.692 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:31.688+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:31.706 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:31.702+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:31.898 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:31.894+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:32.215 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:32.209+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:32.215 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:32.211+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:32.333 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:32.328+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:32.651 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:32.646+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:32.656 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:32.651+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:32.684 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:32.679+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:32.759 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:32.755+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:32.935 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:32.929+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:33.223 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:33.218+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:33.227 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:33.221+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:33.303 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:33.298+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:33.655 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:33.650+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:33.656 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:33.650+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:33.665 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:33.659+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:33.732 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:33.726+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:33.927 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:33.923+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:34.186 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:34.182+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:34.188 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:34.185+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:34.348 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:34.342+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:34.630 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:34.625+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:34.649 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:34.644+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:34.672 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:34.667+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:34.674 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:34.670+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:34.752 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:34.747+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:34.931 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:34.925+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:35.148 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:35.143+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:35.148 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:35.144+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:35.305 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:35.299+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:35.618 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:35.613+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:35.665 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:35.660+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:35.678 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:35.674+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:35.682 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:35.677+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:35.777 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:35.772+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:35.943 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:35.938+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:36.188 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:36.183+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:36.191 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:36.187+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:36.291 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:36.285+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:36.637 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:36.632+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:36.641 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:36.636+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:36.643 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:36.637+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:36.698 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:36.693+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:36.783 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:36.778+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:36.962 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:36.958+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:37.206 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:37.202+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:37.209 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:37.205+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:37.333 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:37.328+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:37.665 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:37.661+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:37.684 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:37.678+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:37.685 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:37.679+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:37.718 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:37.713+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:37.799 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:37.793+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:37.935 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:37.927+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:38.239 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:38.234+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:38.241 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:38.237+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:38.311 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:38.305+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:38.655 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:38.649+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:38.655 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:38.650+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:38.693 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:38.688+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:38.736 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:38.731+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:38.830 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:38.824+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:38.890 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:38.885+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:39.220 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:39.215+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:39.221 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:39.216+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:39.323 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:39.318+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:39.639 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:39.632+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:39.639 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:39.634+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:39.749 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:39.744+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:39.776 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:39.771+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:39.867 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:39.861+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:39.928 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:39.923+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:40.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:40.198+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:40.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:40.199+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:40.336 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:40.329+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:40.620 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:40.614+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:40.621 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:40.616+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:40.769 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:40.765+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:40.807 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:40.802+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:40.913 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:40.909+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:40.920 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:40.915+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:41.168 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:41.162+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:41.169 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:41.164+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:41.310 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:41.304+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:41.600 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:41.595+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:41.603 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:41.598+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:41.744 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:41.740+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:41.785 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:41.780+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:41.894 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:41.889+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:41.953 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:41.948+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:42.141 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:42.133+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:42.142 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:42.135+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:42.342 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:42.336+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:42.652 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:42.647+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:42.653 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:42.649+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:42.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:42.759+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:42.799 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:42.793+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:42.896 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:42.891+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:42.920 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:42.916+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:43.151 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:43.146+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:43.152 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:43.147+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:43.328 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:43.322+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:43.704 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:43.698+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:43.707 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:43.702+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:43.786 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:43.780+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:43.832 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:43.826+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:43.948 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:43.942+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:43.961 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:43.957+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:44.199 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:44.194+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:44.200 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:44.195+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:44.357 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:44.350+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:44.684 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:44.678+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:44.685 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:44.680+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:44.809 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:44.805+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:44.836 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:44.830+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:44.945 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:44.940+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:45.018 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:45.013+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:45.207 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:45.202+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:45.209 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:45.205+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:45.389 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:45.383+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:45.694 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:45.689+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:45.696 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:45.692+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:45.799 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:45.793+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:45.827 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:45.822+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:45.920 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:45.916+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:46.025 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:46.020+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:46.179 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:46.175+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:46.179 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:46.176+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:46.423 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:46.417+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:46.661 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:46.656+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:46.661 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:46.656+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:46.812 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:46.805+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:46.825 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:46.819+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:46.959 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:46.954+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:47.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:47.060+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:47.204 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:47.199+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:47.204 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:47.200+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:47.434 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:09:47.425+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 2 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 44sec e433 v433) 2019-11-25T17:09:47.444 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:47.438+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:47.621 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:47.616+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:47.622 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:47.617+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:47.804 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:47.799+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:47.840 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:47.835+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:47.919 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:47.914+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:48.108 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:48.103+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:48.206 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:48.200+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:48.207 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:48.202+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:48.465 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:48.460+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:48.591 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:48.585+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:48.592 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:48.587+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:48.804 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:48.799+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:48.833 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:48.828+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:48.923 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:48.919+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:49.145 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:49.140+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:49.235 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:49.229+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:49.236 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:49.229+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:49.487 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:49.482+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:49.619 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:49.614+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:49.622 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:49.618+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:49.812 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:49.807+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:49.878 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:49.873+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:49.902 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:49.898+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:50.148 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:50.143+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:50.209 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:50.203+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:50.210 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:50.206+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:50.457 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:50.453+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:50.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:50.600+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:50.605 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:50.600+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:50.801 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:50.796+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:50.913 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:50.908+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:50.947 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:50.942+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:51.176 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:51.170+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:51.190 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:51.185+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:51.191 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:51.186+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:51.438 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:51.432+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:51.589 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:51.583+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:51.589 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:51.585+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:51.836 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:51.830+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:51.890 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:51.886+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:51.969 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:51.964+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:52.218 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:52.212+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:52.219 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:52.213+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:52.220 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:52.215+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:52.424 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:52.419+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:52.501 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:09:52.497+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 44sec e433 v433) 2019-11-25T17:09:52.613 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:52.610+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:52.615 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:52.611+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:52.833 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:52.829+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:52.914 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:52.909+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:53.023 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:53.018+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:53.191 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:53.185+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:53.191 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:53.187+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:53.197 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:53.192+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:53.444 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:53.439+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:53.576 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:53.572+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:53.577 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:53.572+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:53.880 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:53.875+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:53.936 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:53.932+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:54.029 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:54.024+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:54.236 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:54.232+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:54.248 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:54.243+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:54.251 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:54.245+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:54.426 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:54.421+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:54.626 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:54.622+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:54.628 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:54.623+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:54.871 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:54.867+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:54.925 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:54.921+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:54.990 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:54.984+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:55.257 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:55.252+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:55.258 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:55.253+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:55.293 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:55.288+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:55.400 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:55.396+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:55.663 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:55.657+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:55.664 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:55.659+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:55.921 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:55.915+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:55.925 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:55.920+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:55.958 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:55.953+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:56.297 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:56.293+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:56.300 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:56.296+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:56.322 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:56.317+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:56.374 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:56.369+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:56.706 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:56.701+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:56.707 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:56.702+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:56.947 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:56.942+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:56.965 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:56.960+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:56.997 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:56.992+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:57.267 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:57.261+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:57.268 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:57.263+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:57.297 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:57.293+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:57.424 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:57.419+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:57.517 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:09:57.510+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 44sec e433 v433) 2019-11-25T17:09:57.743 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:57.737+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:57.743 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:57.739+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:57.958 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:57.953+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:57.986 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:57.981+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:58.006 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:58.002+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:58.274 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:58.270+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:58.298 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:58.294+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:58.299 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:58.294+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:58.423 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:58.419+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:58.709 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:58.704+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:58.710 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:58.705+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:58.960 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:58.955+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:58.984 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:58.978+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:09:58.984 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:58.979+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:59.300 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:09:59.296+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:09:59.329 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:59.323+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:09:59.330 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:09:59.325+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:09:59.408 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:09:59.404+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:09:59.737 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:59.733+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:09:59.740 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:09:59.737+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:09:59.947 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:09:59.942+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:09:59.983 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:09:59.978+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:09:59.994 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:09:59.989+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:00.351 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:00.345+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:00.359 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:00.354+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:00.359 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:00.355+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:00.377 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:00.372+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:00.783 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:00.776+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:00.784 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:00.778+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:00.911 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:00.906+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:00.988 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:00.982+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:01.002 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:00.996+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:01.344 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:01.339+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:01.350 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:01.344+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:01.368 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:01.363+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:01.371 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:01.367+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:01.774 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:01.767+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:01.774 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:01.768+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:01.957 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:01.951+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:01.968 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:01.962+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:02.006 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:02.001+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:02.349 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:02.343+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:02.385 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:02.381+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:02.403 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:02.398+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:02.404 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:02.399+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:02.531 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:10:02.524+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 44sec e433 v433) 2019-11-25T17:10:02.731 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:02.725+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:02.733 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:02.728+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:02.956 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:02.951+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:02.958 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:55:07.541 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:02.965 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:55:07.548 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:02.969 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:55:07.575 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:02.969 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:55:07.575 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:02.974 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:55:07.580 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:02.975 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:55:07.581 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:02.983 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:02.979+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:02.986 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:02.982+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:03.248 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:55:07.831 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:03.264 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:55:07.870 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:03.299 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:55:07.905 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:10:03.369 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:03.365+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:03.389 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:03.383+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:03.409 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:03.405+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:03.410 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:03.406+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:03.777 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:03.772+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:03.778 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:03.773+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:03.942 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:03.936+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:03.969 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:03.964+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:03.977 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:03.972+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:04.339 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:04.334+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:04.402 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:04.397+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:04.403 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:04.398+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:04.423 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:04.417+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:04.737 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:04.732+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:04.738 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:04.733+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:04.941 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:04.936+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:04.957 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:04.951+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:04.966 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:04.961+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:05.366 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:05.362+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:05.367 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:05.364+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:05.375 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:05.371+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:05.464 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:05.456+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:05.783 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:05.776+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:05.783 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:05.778+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:05.952 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:05.947+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:05.995 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:05.991+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:06.015 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:06.010+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:06.396 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:06.390+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:06.399 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:06.394+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:06.414 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:06.409+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:06.436 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:06.431+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:06.768 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:06.761+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:06.769 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:06.763+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:06.948 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:06.942+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:07.008 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:07.003+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:07.023 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:07.016+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:07.372 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:07.366+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:07.373 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:07.367+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:07.392 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:07.386+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:07.472 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:07.466+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:07.555 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:10:07.544+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 44sec e433 v433) 2019-11-25T17:10:07.817 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:07.812+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:07.818 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:07.813+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:07.929 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:07.923+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:07.979 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:07.973+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:08.061 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:08.055+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:08.383 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:08.377+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:08.384 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:08.378+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:08.438 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:08.432+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:08.505 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:08.500+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:08.845 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:08.839+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:08.846 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:08.841+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:08.928 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:08.922+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:08.986 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:08.979+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:09.095 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:09.089+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:09.357 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:09.351+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:09.359 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:09.354+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:09.445 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:09.439+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:09.516 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:09.510+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:09.833 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:09.827+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:09.836 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:09.831+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:09.923 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:09.918+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:09.939 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:09.934+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:10.114 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:10.109+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:10.327 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:10.322+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:10.329 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:10.324+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:10.422 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:10.417+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:10.514 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:10.509+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:10.865 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:10.858+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:10.865 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:10.860+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:10.945 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:10.937+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:10.951 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:10.944+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:11.133 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:11.123+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:11.362 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:11.356+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:11.363 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:11.358+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:11.387 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:11.378+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:11.513 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:11.507+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:11.878 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:11.872+0000 16949700 -1 osd.4 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:08:54.666669+0000 (oldest deadline 2019-11-25T17:09:34.666669+0000) 2019-11-25T17:10:11.879 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:11.873+0000 16949700 -1 osd.4 433 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:11.899 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:10:11.892+0000 16949700 -1 osd.1 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:32.940934+0000 front 2019-11-25T17:08:33.212327+0000 (oldest deadline 2019-11-25T17:09:16.398123+0000) 2019-11-25T17:10:11.989 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:11.983+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:12.189 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:10:12.183+0000 16949700 -1 osd.7 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:29.780252+0000 front 2019-11-25T17:08:29.786876+0000 (oldest deadline 2019-11-25T17:09:14.419860+0000) 2019-11-25T17:10:12.341 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:10:12.335+0000 16949700 -1 osd.0 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.412254+0000 front 2019-11-25T17:08:32.963387+0000 (oldest deadline 2019-11-25T17:09:16.195068+0000) 2019-11-25T17:10:12.378 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:12.372+0000 16949700 -1 osd.3 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:31.595025+0000 front 2019-11-25T17:08:31.593838+0000 (oldest deadline 2019-11-25T17:09:12.274726+0000) 2019-11-25T17:10:12.379 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:12.373+0000 16949700 -1 osd.3 433 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:12.501 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:10:12.495+0000 16949700 -1 osd.5 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.784974+0000 front 2019-11-25T17:08:28.855189+0000 (oldest deadline 2019-11-25T17:09:12.070189+0000) 2019-11-25T17:10:12.614 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:10:12.604+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 44sec e433 v433) 2019-11-25T17:10:13.020 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:10:13.014+0000 16949700 -1 osd.2 433 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:08:34.785100+0000 front 2019-11-25T17:08:31.669292+0000 (oldest deadline 2019-11-25T17:09:16.340222+0000) 2019-11-25T17:10:13.165 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:10:13.160+0000 16949700 -1 osd.4 434 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e432) 2019-11-25T17:10:13.589 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:13.580+0000 16949700 -1 osd.3 434 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:14.783 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:14.760+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:15.806 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:15.802+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:16.801 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:16.796+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:17.838 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:17.833+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:18.887 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:18.883+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:19.912 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:19.908+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:20.908 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:20.904+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:21.928 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:21.924+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:22.942 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:22.938+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:23.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:23.915+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:24.894 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:24.889+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:25.914 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:25.910+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:26.922 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:26.918+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:27.908 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:27.904+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:28.957 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:28.953+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:10:29.927 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:10:29.923+0000 16949700 -1 osd.3 435 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:121959 6.8 6:12fd8b4f:root::.bucket.meta.%e9%90%8a%ef%84%91%eb%8c%8e%e9%90%8b%eb%8c%8e%e0%a4%82%eb%8c%8e%ef%84%91%e0%a4%81%e8%94%8a%e1%a0%81%e8%94%8a%ed%88%90%eb%8c%8b%ed%88%8d%3a103c5a94-eb69-4977-8c90-9f6d634d8761.4358.660:head [create,call version.set in=58b,writefull 0~319 in=319b,setxattr user.rgw.acl (193) in=205b] snapc 0=[] ondisk+write+known_if_redirected e431) 2019-11-25T17:11:30.504 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:30.498+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:31.466 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:31.462+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:31.541 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:31.536+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:32.500 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:32.494+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:32.508 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:32.502+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:33.415 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:33.409+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:33.481 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:33.475+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:33.519 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:33.513+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:33.521 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:33.515+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:33.560 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:33.555+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:34.466 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:34.460+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:34.498 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:34.493+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:34.528 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:34.523+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:34.546 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:34.542+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:34.612 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:34.607+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:34.693 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:34.687+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:35.468 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:35.462+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:35.473 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:35.467+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:35.511 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:35.506+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:35.539 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:35.534+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:35.634 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:35.629+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:35.710 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:35.704+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:36.429 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:36.425+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:36.483 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:36.475+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:36.502 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:36.498+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:36.593 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:36.583+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:36.597 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:36.592+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:36.697 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:36.692+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:37.402 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:37.396+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:37.446 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:37.441+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:37.478 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:37.473+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:37.604 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:37.600+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:37.621 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:37.616+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:37.714 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:37.709+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:38.401 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:38.395+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:38.436 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:38.431+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:38.521 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:38.515+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:38.579 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:38.574+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:38.594 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:38.588+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:38.756 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:38.751+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:39.396 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:39.390+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:39.397 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:39.393+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:39.526 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:39.521+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:39.558 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:39.554+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:39.593 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:39.588+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:39.782 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:39.777+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:40.380 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:40.374+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:40.415 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:40.410+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:40.562 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:40.558+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:40.576 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:40.572+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:40.587 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:40.582+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:40.751 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:40.746+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:41.339 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:41.334+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:41.398 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:41.394+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:41.589 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:41.584+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:41.599 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:41.594+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:41.609 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:41.603+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:41.768 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:41.762+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:42.317 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:42.313+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:42.377 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:42.372+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:42.561 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:42.552+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:42.623 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:42.617+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:42.633 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:42.628+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:42.730 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:42.723+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:43.288 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:43.282+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:43.373 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:43.367+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:43.582 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:43.576+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:43.619 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:43.614+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:43.635 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:43.629+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:43.706 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:43.701+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:44.306 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:44.301+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:44.420 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:44.413+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:44.583 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:44.577+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:44.585 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:44.580+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:44.603 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:44.598+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:44.679 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:44.672+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:45.272 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:45.267+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:45.460 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:45.453+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:45.606 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:45.601+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:45.612 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:45.606+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:45.685 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:45.679+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:45.711 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:45.705+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:46.256 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:46.251+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:46.420 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:46.415+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:46.621 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:46.616+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:46.642 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:46.635+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:46.712 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:46.704+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:46.754 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:46.748+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:47.284 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:47.279+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:47.406 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:47.400+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:47.618 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:47.612+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:47.624 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:47.619+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:47.695 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:47.689+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:47.751 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:47.745+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:48.323 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:48.318+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:48.463 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:48.457+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:48.623 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:48.617+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:48.645 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:48.640+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:48.665 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:48.659+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:48.752 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:48.747+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:48.755 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:48.751+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:49.338 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:49.333+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:49.479 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:49.473+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:49.632 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:49.625+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:49.685 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:49.679+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:49.685 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:49.679+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:49.775 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:49.770+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:49.779 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:49.775+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:50.319 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:50.314+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:50.467 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:50.461+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:50.644 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:50.639+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:50.658 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:50.653+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:50.692 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:50.686+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:50.738 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:50.734+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:50.739 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:50.735+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:51.280 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:51.275+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:51.444 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:51.438+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:51.673 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:51.667+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:51.685 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:51.679+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:51.717 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:51.709+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:51.770 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:51.765+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:51.771 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:51.767+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:52.238 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:52.234+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:52.479 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:52.472+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:52.676 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:52.670+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:52.695 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:52.689+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:52.738 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:52.732+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:52.816 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:52.809+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:52.818 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:52.811+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:53.216 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:53.209+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:53.514 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:53.509+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:53.691 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:53.686+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:53.704 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:53.698+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:53.733 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:53.726+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:53.853 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:53.847+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:53.854 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:53.848+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:54.230 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:54.225+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:54.555 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:54.548+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:54.658 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:54.651+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:54.679 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:54.674+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:54.736 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:54.731+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:54.847 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:54.841+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:54.848 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:54.843+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:55.237 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:55.231+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:55.532 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:55.526+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:55.616 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:55.610+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:55.721 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:55.716+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:55.744 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:55.738+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:55.848 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:55.843+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:55.850 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:55.845+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:56.231 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:56.225+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:56.574 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:56.569+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:56.603 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:56.596+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:56.727 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:56.720+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:56.750 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:56.743+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:56.852 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:56.839+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:56.852 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:56.841+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:57.261 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:57.255+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:57.582 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:57.576+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:57.636 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:57.630+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:57.739 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:57.732+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:57.772 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:57.766+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:57.819 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:57.813+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:57.821 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:57.816+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:58.276 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:58.270+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:58.601 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:58.595+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:58.624 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:58.617+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:58.728 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:58.714+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:58.780 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:58.773+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:11:58.782 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:58.777+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:58.785 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:58.781+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:59.279 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:11:59.273+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:11:59.620 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:11:59.615+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:11:59.634 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:11:59.628+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:11:59.679 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:11:59.673+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:11:59.753 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:59.747+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:11:59.753 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:11:59.750+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:11:59.817 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:11:59.811+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:00.295 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:00.290+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:00.593 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:00.587+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:00.607 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:00.601+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:00.690 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:00.684+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:00.713 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:00.707+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:00.714 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:00.708+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:00.834 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:00.828+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:01.337 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:01.331+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:01.577 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:01.572+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:01.658 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:01.652+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:01.682 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:01.677+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:01.691 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:01.686+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:01.694 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:01.690+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:01.789 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:01.784+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:02.344 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:02.338+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:02.571 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:02.566+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:02.710 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:02.704+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:02.728 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:02.722+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:02.728 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:02.723+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:02.731 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:02.727+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:02.755 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:02.750+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:03.365 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:03.360+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:03.528 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:03.523+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:03.709 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:03.701+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:03.725 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:03.720+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:03.728 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:03.724+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:03.747 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:03.740+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:03.769 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:03.762+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:04.408 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:04.402+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:04.576 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:04.571+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:04.693 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:04.686+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:04.693 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:04.689+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:04.710 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:04.702+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:04.732 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:04.727+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:04.817 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:04.811+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:05.420 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:05.414+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:05.548 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:05.542+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:05.671 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:05.666+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:05.672 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:05.667+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:05.674 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:05.670+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:05.734 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:05.728+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:05.842 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:05.836+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:06.388 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:06.382+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:06.578 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:06.572+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:06.675 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:06.670+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:06.715 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:06.711+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:06.717 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:06.713+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:06.725 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:06.719+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:06.845 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:06.838+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:07.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:07.348+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:07.619 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:07.614+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:07.715 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:07.709+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:07.715 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:07.710+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:07.727 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:07.721+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:07.761 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:07.755+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:07.834 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:07.827+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:08.017 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:12:08.010+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 42sec e438 v438) 2019-11-25T17:12:08.348 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:08.343+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:08.626 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:08.620+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:08.739 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:08.730+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:08.739 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:08.731+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:08.739 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:08.734+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:08.766 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:08.760+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:08.792 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:08.786+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:09.306 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:09.301+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:09.677 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:09.670+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:09.718 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:09.712+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:09.720 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:09.713+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:09.763 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:09.755+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:09.800 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:09.795+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:09.834 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:09.828+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:10.330 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:10.324+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:10.648 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:10.640+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:10.680 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:10.675+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:10.683 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:10.679+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:10.777 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:10.771+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:10.812 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:10.806+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:10.843 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:10.837+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:11.296 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:11.291+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:11.628 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:11.622+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:11.686 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:11.681+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:11.687 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:11.682+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:11.799 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:11.793+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:11.820 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:11.815+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:11.828 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:11.822+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:12.344 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:12.339+0000 16949700 -1 osd.3 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.287618+0000 front 2019-11-25T17:10:51.273642+0000 (oldest deadline 2019-11-25T17:11:32.510154+0000) 2019-11-25T17:12:12.662 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:12:12.656+0000 16949700 -1 osd.0 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.420189+0000 front 2019-11-25T17:10:50.341487+0000 (oldest deadline 2019-11-25T17:11:30.995295+0000) 2019-11-25T17:12:12.662 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:12.656+0000 16949700 -1 osd.2 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:48.097906+0000 front 2019-11-25T17:10:47.212507+0000 (oldest deadline 2019-11-25T17:11:32.407746+0000) 2019-11-25T17:12:12.663 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:12.658+0000 16949700 -1 osd.2 438 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:12.838 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:12:12.833+0000 16949700 -1 osd.6 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:49.419597+0000 front 2019-11-25T17:10:50.696231+0000 (oldest deadline 2019-11-25T17:11:33.727089+0000) 2019-11-25T17:12:12.842 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:12:12.835+0000 16949700 -1 osd.7 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.754810+0000 front 2019-11-25T17:10:51.763367+0000 (oldest deadline 2019-11-25T17:11:33.392254+0000) 2019-11-25T17:12:12.848 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:12:12.843+0000 16949700 -1 osd.5 438 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:10:51.813182+0000 front 2019-11-25T17:10:45.754321+0000 (oldest deadline 2019-11-25T17:11:29.910600+0000) 2019-11-25T17:12:13.092 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:12:13.084+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 42sec e438 v438) 2019-11-25T17:12:13.886 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:13.866+0000 16949700 -1 osd.2 439 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:14.880 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:14.875+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:15.878 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:15.874+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:16.921 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:16.917+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:17.914 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:17.910+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:18.492 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:57:23.074 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.521 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:57:23.104 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.555 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:57:23.161 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.559 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:57:23.165 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.564 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:57:23.170 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.565 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:57:23.171 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.713 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:57:23.295 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.759 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:57:23.365 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.800 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:57:23.405 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:12:18.910 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:18.906+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:19.955 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:19.951+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:20.967 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:20.963+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:21.966 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:21.961+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:23.010 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:23.006+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:24.021 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:24.015+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:24.992 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:24.988+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:25.992 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:25.987+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:27.003 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:26.999+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:27.998 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:27.994+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:28.972 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:28.967+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:29.989 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:29.984+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:31.037 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:12:31.032+0000 16949700 -1 osd.2 440 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122215 2.32s0 2:4f7a1e3f:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_dealde%252Fl04 011e%25204c8df%2fflash.png:head [getxattrs,stat,read 0~4194304] snapc 0=[] ondisk+read+known_if_redirected e438) 2019-11-25T17:12:47.539 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:47.522+0000 39278700 -1 osd.3 442 build_incremental_map_msg missing incremental map 442 2019-11-25T17:12:47.630 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:47.620+0000 39278700 -1 osd.3 442 build_incremental_map_msg missing incremental map 442 2019-11-25T17:12:47.836 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:47.625+0000 3a27a700 -1 osd.3 442 build_incremental_map_msg missing incremental map 442 2019-11-25T17:12:47.960 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:12:47.851+0000 35a71700 -1 osd.3 442 build_incremental_map_msg missing incremental map 442 2019-11-25T17:13:00.203 INFO:teuthology.orchestra.run.smithi153.stderr:testCopy (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:13:03.869 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:58:08.452 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:03.895 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:58:08.478 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:03.905 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:58:08.511 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:03.951 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:58:08.557 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:03.952 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:58:08.557 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:03.954 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:58:08.560 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:04.002 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:58:08.585 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:04.016 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:58:08.622 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:04.038 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:58:08.643 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:19.157 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:19.152+0000 16949700 -1 osd.4 443 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:20.133 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:20.129+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:21.168 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:21.163+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:22.179 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:22.174+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:23.193 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:23.187+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:24.189 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:24.184+0000 16949700 -1 osd.4 443 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:25.207 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:25.202+0000 16949700 -1 osd.4 443 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:26.187 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:26.179+0000 16949700 -1 osd.4 443 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:27.145 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:27.139+0000 16949700 -1 osd.4 443 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:28.155 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:28.150+0000 16949700 -1 osd.4 443 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:29.160 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:29.155+0000 16949700 -1 osd.4 443 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:30.213 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:30.208+0000 16949700 -1 osd.4 443 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:31.238 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:31.233+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:32.233 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:32.227+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:33.281 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:33.276+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:34.260 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:34.254+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:35.253 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:35.248+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:36.238 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:36.232+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:37.208 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:37.202+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:38.255 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:38.249+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:39.290 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:39.285+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:40.304 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:40.294+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:41.276 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:41.272+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:42.244 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:42.240+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:43.217 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:43.212+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:44.250 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:44.245+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:45.287 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:45.281+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:46.260 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:46.255+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:47.263 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:47.257+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:48.238 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:48.233+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:49.273 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:49.268+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:50.253 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:50.248+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:51.245 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:51.240+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:51.700 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:51.695+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:52.246 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:52.241+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:52.735 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:52.730+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:53.235 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:53.230+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:53.455 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:13:53.450+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:13:53.683 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:13:53.676+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:13:53.712 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:53.707+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:53.997 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:58:58.580 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.007 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:58:58.590 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.022 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:58:58.622 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.022 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:58:58.626 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.022 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:58:58.624 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.023 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:58:58.629 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.079 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:07:58:58.661 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.105 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:07:58:58.708 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.106 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:07:58:58.708 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:13:54.230 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:54.225+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:13:54.231 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:54.226+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:54.501 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:13:54.496+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:13:54.688 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:54.681+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:54.731 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:13:54.727+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:13:54.889 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:13:54.884+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:13:55.254 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:55.248+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:13:55.254 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:55.249+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:55.496 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:13:55.492+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:13:55.516 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:13:55.501+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:13:55.703 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:13:55.699+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:13:55.726 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:55.722+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:55.858 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:13:55.853+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:13:56.171 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:13:56.164+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:13:56.291 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:56.284+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:13:56.293 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:56.286+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:56.468 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:13:56.463+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:13:56.532 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:13:56.526+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:13:56.717 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:56.710+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:56.754 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:13:56.748+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:13:56.857 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:13:56.844+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:13:57.130 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:13:57.123+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:13:57.254 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:57.247+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:13:57.256 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:57.249+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:57.515 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:13:57.510+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:13:57.519 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:13:57.514+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:13:57.712 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:57.707+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:57.800 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:13:57.793+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:13:57.847 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:13:57.841+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:13:58.147 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:13:58.140+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:13:58.285 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:58.280+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:13:58.288 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:58.283+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:58.483 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:13:58.479+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:13:58.518 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:13:58.512+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:13:58.718 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:58.713+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:58.789 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:13:58.783+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:13:58.852 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:13:58.847+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:13:59.124 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:13:59.118+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:13:59.277 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:59.270+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:13:59.278 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:13:59.272+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:13:59.506 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:13:59.499+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:13:59.535 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:13:59.530+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:13:59.721 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:13:59.717+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:13:59.762 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:13:59.758+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:13:59.826 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:13:59.820+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:00.091 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:00.081+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:00.291 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:00.285+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:00.292 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:00.287+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:00.498 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:00.493+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:00.526 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:00.521+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:00.691 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:00.686+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:00.735 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:00.730+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:00.853 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:00.847+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:01.124 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:01.119+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:01.270 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:01.264+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:01.271 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:01.266+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:01.519 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:01.515+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:01.536 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:01.531+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:01.662 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:01.657+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:01.774 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:01.770+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:01.832 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:01.828+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:02.137 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:02.132+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:02.270 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:02.264+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:02.270 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:02.266+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:02.535 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:02.528+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:02.558 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:02.552+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:02.662 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:02.656+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:02.749 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:02.743+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:02.863 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:02.858+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:03.179 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:03.176+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:03.295 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:03.291+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:03.299 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:03.294+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:03.505 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:03.500+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:03.581 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:03.576+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:03.683 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:03.678+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:03.723 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:03.718+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:03.915 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:03.911+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:04.205 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:04.201+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:04.345 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:04.341+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:04.348 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:04.344+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:04.537 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:04.532+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:04.567 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:04.561+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:04.648 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:04.642+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:04.772 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:04.766+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:04.890 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:04.885+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:05.177 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:05.172+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:05.325 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:05.319+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:05.326 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:05.321+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:05.542 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:05.537+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:05.565 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:05.560+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:05.698 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:05.693+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:05.745 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:05.740+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:05.934 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:05.929+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:06.185 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:06.180+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:06.305 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:06.300+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:06.308 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:06.304+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:06.511 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:06.506+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:06.550 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:06.546+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:06.719 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:06.714+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:06.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:06.756+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:06.988 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:06.983+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:07.173 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:07.168+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:07.304 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:07.300+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:07.307 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:07.303+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:07.502 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:07.495+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:07.517 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:07.512+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:07.720 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:07.715+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:07.768 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:07.762+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:07.941 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:07.936+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:08.149 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:08.144+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:08.326 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:08.320+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:08.327 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:08.322+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:08.461 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:08.457+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:08.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:08.470+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:08.755 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:08.750+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:08.774 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:08.768+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:08.923 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:08.918+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:09.182 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:09.177+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:09.315 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:09.303+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:09.318 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:09.310+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:09.417 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:09.411+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:09.462 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:09.457+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:09.772 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:09.768+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:09.800 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:09.795+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:09.950 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:09.944+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:10.179 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:10.174+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:10.367 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:10.362+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:10.368 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:10.363+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:10.424 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:10.418+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:10.449 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:10.445+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:10.826 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:10.821+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:10.827 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:10.821+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:10.962 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:10.958+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:11.142 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:11.137+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:11.386 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:11.381+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:11.387 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:11.382+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:11.415 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:11.410+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:11.427 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:11.423+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:11.833 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:11.828+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:11.853 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:11.847+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:11.940 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:11.933+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:12.175 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:12.170+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:12.340 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:12.335+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:12.343 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:12.339+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:12.467 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:12.462+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:12.468 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:12.464+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:12.808 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:12.803+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:12.871 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:12.866+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:12.966 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:12.961+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:13.146 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:13.140+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:13.317 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:13.311+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:13.318 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:13.313+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:13.455 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:13.449+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:13.498 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:13.493+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:13.841 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:13.837+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:13.896 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:13.891+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:13.957 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:13.952+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:14.141 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:14.135+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:14.324 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:14.319+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:14.326 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:14.322+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:14.421 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:14.416+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:14.517 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:14.512+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:14.855 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:14.850+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:14.907 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:14.901+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:14.954 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:14.948+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:15.162 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:15.156+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:15.280 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:15.274+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:15.280 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:15.275+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:15.453 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:15.446+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:15.524 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:15.519+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:15.888 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:15.882+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:15.888 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:15.883+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:15.929 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:15.924+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:16.198 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:16.192+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:16.311 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:16.304+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:16.311 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:16.306+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:16.485 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:16.479+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:16.507 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:16.500+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:16.889 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:16.882+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:16.899 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:16.894+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:16.960 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:16.954+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:17.229 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:17.223+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:17.344 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:17.338+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:17.347 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:17.342+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:17.441 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:17.435+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:17.551 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:17.546+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:17.850 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:17.846+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:17.941 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:17.936+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:17.983 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:17.978+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:18.210 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:18.204+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:18.381 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:18.376+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:18.385 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:18.380+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:18.474 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:18.468+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:18.589 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:18.584+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:18.811 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:18.806+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:18.959 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:18.954+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:19.001 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:18.996+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:19.176 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:19.170+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:19.413 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:19.407+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:19.414 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:19.408+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:19.505 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:19.500+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:19.579 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:19.574+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:19.849 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:19.844+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:19.911 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:19.907+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:20.032 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:20.027+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:20.225 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:20.220+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:20.402 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:20.395+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:20.403 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:20.397+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:20.535 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:20.529+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:20.626 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:20.621+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:20.816 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:20.811+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:20.900 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:20.894+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:21.075 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:21.069+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:21.251 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:21.245+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:21.419 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:21.414+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:21.421 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:21.417+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:21.578 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:21.572+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:21.656 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:21.651+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:21.805 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:21.800+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:21.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:21.914+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:22.082 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:22.077+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:22.243 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:22.238+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:22.471 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:22.465+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:22.472 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:22.467+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:22.570 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:22.563+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:22.614 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:22.609+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:22.798 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:22.793+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:22.880 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:14:22.875+0000 16949700 -1 osd.3 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.420927+0000 front 2019-11-25T17:13:11.414446+0000 (oldest deadline 2019-11-25T17:13:51.466098+0000) 2019-11-25T17:14:23.137 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:23.132+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:13:14.439719+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:23.211 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:14:23.206+0000 16949700 -1 osd.5 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:12.094765+0000 front 2019-11-25T17:13:14.151342+0000 (oldest deadline 2019-11-25T17:13:55.603922+0000) 2019-11-25T17:14:23.463 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:23.456+0000 16949700 -1 osd.4 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:11.429276+0000 front 2019-11-25T17:13:11.430750+0000 (oldest deadline 2019-11-25T17:13:54.005977+0000) 2019-11-25T17:14:23.463 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:23.458+0000 16949700 -1 osd.4 443 get_health_metrics reporting 5 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:23.540 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:14:23.534+0000 16949700 -1 osd.7 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.871103+0000 front 2019-11-25T17:13:13.804543+0000 (oldest deadline 2019-11-25T17:13:53.316989+0000) 2019-11-25T17:14:23.639 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:14:23.634+0000 16949700 -1 osd.0 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:09.794760+0000 front 2019-11-25T17:13:09.802900+0000 (oldest deadline 2019-11-25T17:13:54.708084+0000) 2019-11-25T17:14:23.663 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:14:23.567+0000 38a77700 -1 osd.6 443 build_incremental_map_msg missing incremental map 442 2019-11-25T17:14:23.811 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:14:23.805+0000 16949700 -1 osd.1 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:04.229232+0000 front 2019-11-25T17:13:10.094492+0000 (oldest deadline 2019-11-25T17:13:52.662368+0000) 2019-11-25T17:14:24.126 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:14:24.122+0000 16949700 -1 osd.2 443 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:13:13.818528+0000 front 2019-11-25T17:14:23.656120+0000 (oldest deadline 2019-11-25T17:13:54.294867+0000) 2019-11-25T17:14:24.672 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:14:24.651+0000 16949700 -1 osd.4 444 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e442) 2019-11-25T17:14:29.537 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:14:29.530+0000 34a6f700 -1 osd.6 448 build_incremental_map_msg missing incremental map 448 2019-11-25T17:14:29.892 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:14:29.846+0000 36a73700 -1 osd.6 448 build_incremental_map_msg missing incremental map 448 2019-11-25T17:14:29.936 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:14:29.883+0000 36a73700 -1 osd.6 448 build_incremental_map_msg missing incremental map 448 2019-11-25T17:14:30.332 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:14:30.101+0000 36a73700 -1 osd.6 448 build_incremental_map_msg missing incremental map 448 2019-11-25T17:14:30.432 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:14:30.353+0000 33a6d700 -1 osd.6 448 build_incremental_map_msg missing incremental map 448 2019-11-25T17:14:30.972 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:14:30.927+0000 3a27a700 -1 osd.6 448 build_incremental_map_msg missing incremental map 448 2019-11-25T17:14:59.444 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:14:59.440+0000 16949700 -1 osd.6 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122576 3.25 3:a76cbc4c:::.dir.103c5a94-eb69-4977-8c90-9f6d634d8761.4358.661:head [create,call rgw.bucket_init_index in=20b] snapc 0=[] ondisk+write+known_if_redirected e447) 2019-11-25T17:15:29.858 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:29.852+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:30.910 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:30.904+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:31.307 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:31.303+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:31.792 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:31.787+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:31.878 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:31.872+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:32.314 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:32.308+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:32.759 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:32.753+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:32.849 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:32.844+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:33.251 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:33.247+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:33.338 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:33.332+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:33.389 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:33.383+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:33.391 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:33.386+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:33.769 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:33.764+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:33.772 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:33.767+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:33.880 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:33.875+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:34.237 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:34.231+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:34.335 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:34.329+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:34.357 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:34.346+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:34.428 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:34.423+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:34.726 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:34.720+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:34.729 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:34.724+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:34.850 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:34.844+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:35.294 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:35.288+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:35.327 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:35.323+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:35.381 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:35.375+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:35.438 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:35.433+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:35.693 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:35.686+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:35.694 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:35.689+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:35.851 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:35.845+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:36.265 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:36.260+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:36.349 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:36.345+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:36.391 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:36.387+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:36.405 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:36.399+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:36.727 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:36.722+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:36.730 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:36.725+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:36.893 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:36.888+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:37.281 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:37.276+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:37.317 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:37.311+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:37.402 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:37.395+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:37.419 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:37.414+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:37.755 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:37.749+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:37.756 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:37.751+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:37.922 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:37.916+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:38.291 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:38.285+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:38.303 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:38.298+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:38.379 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:38.372+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:38.457 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:38.452+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:38.803 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:38.797+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:38.804 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:38.798+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:38.885 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:38.880+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:39.294 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:39.289+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:39.333 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:39.328+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:39.352 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:39.346+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:39.458 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:39.453+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:39.761 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:39.754+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:39.761 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:39.756+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:39.921 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:39.914+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:40.326 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:40.320+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:40.342 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:40.336+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:40.381 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:40.376+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:40.488 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:40.482+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:40.730 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:40.724+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:40.730 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:40.725+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:40.902 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:40.895+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:41.312 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:41.307+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:41.356 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:41.351+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:41.404 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:41.400+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:41.481 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:41.476+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:41.688 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:41.682+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:41.688 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:41.684+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:41.901 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:41.895+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:42.307 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:42.302+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:42.316 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:42.310+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:42.407 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:42.402+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:42.515 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:42.510+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:42.675 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:42.669+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:42.676 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:42.671+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:42.897 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:42.891+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:43.282 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:43.277+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:43.334 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:43.328+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:43.389 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:43.384+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:43.502 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:43.496+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:43.635 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:43.631+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:43.638 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:43.634+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:43.910 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:43.904+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:44.320 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:44.315+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:44.377 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:44.372+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:44.383 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:44.378+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:44.462 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:44.457+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:44.618 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:44.613+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:44.619 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:44.614+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:44.918 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:44.914+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:45.274 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:45.269+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:45.347 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:45.342+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:45.427 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:45.422+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:45.444 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:45.439+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:45.662 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:45.658+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:45.664 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:45.661+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:45.965 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:45.960+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:46.247 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:46.243+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:46.394 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:46.389+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:46.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:46.471+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:46.486 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:46.481+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:46.659 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:46.655+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:46.660 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:46.656+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:46.986 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:46.981+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:47.212 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:47.206+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:47.396 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:47.390+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:47.443 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:47.438+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:47.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:47.470+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:47.642 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:47.637+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:47.643 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:47.639+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:48.016 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:48.011+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:48.206 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:48.202+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:48.400 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:48.394+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:48.403 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:48.397+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:48.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:48.442+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:48.685 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:48.680+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:48.686 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:48.682+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:49.059 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:49.054+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:49.243 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:49.238+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:49.394 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:49.390+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:49.396 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:49.391+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:49.488 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:49.483+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:49.642 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:49.637+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:49.643 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:49.639+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:50.017 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:50.011+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:50.199 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:50.194+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:50.423 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:50.418+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:50.440 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:50.435+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:50.502 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:50.497+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:50.662 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:50.658+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:50.664 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:50.660+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:50.979 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:50.970+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:51.244 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:51.239+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:51.403 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:51.398+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:51.405 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:51.399+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:51.537 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:51.532+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:51.647 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:51.641+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:51.648 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:51.643+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:51.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:51.960+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:52.229 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:52.224+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:52.383 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:52.378+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:52.398 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:52.393+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:52.525 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:52.519+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:52.674 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:52.669+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:52.674 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:52.671+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:52.953 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:52.943+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:53.217 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:53.212+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:53.419 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:53.413+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:53.430 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:53.425+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:53.494 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:53.488+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:53.652 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:53.648+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:53.655 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:53.651+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:53.977 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:53.972+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:54.179 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:54.174+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:54.410 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:54.405+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:54.466 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:54.461+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:54.540 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:54.535+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:54.648 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:54.642+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:54.653 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:54.644+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:55.008 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:55.002+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:55.185 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:55.179+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:55.392 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:55.386+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:55.499 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:55.494+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:55.521 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:55.516+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:55.683 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:55.678+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:55.686 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:55.681+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:55.965 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:55.960+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:56.186 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:56.180+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:56.380 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:56.376+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:56.478 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:56.473+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:56.511 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:56.506+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:56.668 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:56.662+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:56.670 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:56.666+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:57.004 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:56.999+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:57.234 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:57.228+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:57.342 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:57.337+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:57.513 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:57.507+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:57.548 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:57.543+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:57.684 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:57.678+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:57.685 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:57.680+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:57.973 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:57.967+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:58.284 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:58.278+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:58.319 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:58.313+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:58.500 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:58.494+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:58.592 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:58.586+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:58.718 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:58.711+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:58.719 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:58.713+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:58.936 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:58.932+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:15:59.270 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:15:59.263+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:15:59.306 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:15:59.301+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:15:59.506 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:15:59.502+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:15:59.549 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:15:59.544+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:15:59.740 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:59.734+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:15:59.741 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:15:59.735+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:15:59.977 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:15:59.973+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:00.307 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:00.300+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:00.318 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:00.313+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:00.483 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:00.478+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:00.603 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:00.598+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:00.758 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:00.753+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:00.759 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:00.754+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:00.946 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:00.940+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:01.272 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:01.266+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:01.284 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:01.278+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:01.492 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:01.486+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:01.619 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:01.615+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:01.749 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:01.743+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:01.752 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:01.747+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:01.921 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:01.917+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:02.281 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:02.274+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:02.290 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:02.284+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:02.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:02.440+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:02.585 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:02.579+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:02.751 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:02.746+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:02.752 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:02.747+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:02.962 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:02.957+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:03.263 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:03.257+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:03.268 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:03.264+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:03.496 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:03.491+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:03.578 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:03.574+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:03.758 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:03.751+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:03.759 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:03.753+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:03.969 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:03.964+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:04.229 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:04.223+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:04.236 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:04.231+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:04.385 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:16:04.376+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 3 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 45sec e448 v448) 2019-11-25T17:16:04.491 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:04.487+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:04.571 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:04.565+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:04.746 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:04.741+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:04.749 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:04.744+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:04.954 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:04.948+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:05.257 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:05.252+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:05.274 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:05.268+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:05.489 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:05.485+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:05.545 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:05.540+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:05.748 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:05.741+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:05.748 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:05.743+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:06.002 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:05.997+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:06.228 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:06.221+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:06.306 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:06.301+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:06.488 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:06.483+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:06.521 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:06.515+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:06.733 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:06.726+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:06.733 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:06.728+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:07.004 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:06.999+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:07.191 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:07.186+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:07.338 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:07.332+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:07.491 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:07.486+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:07.546 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:07.541+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:07.740 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:07.734+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:07.742 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:07.737+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:08.045 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:08.039+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:08.200 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:08.193+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:08.319 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:08.313+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:08.524 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:08.518+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:08.592 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:08.587+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:08.745 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:08.739+0000 16949700 -1 osd.5 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.655920+0000 front 2019-11-25T17:14:48.340003+0000 (oldest deadline 2019-11-25T17:15:33.692847+0000) 2019-11-25T17:16:08.746 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:08.742+0000 16949700 -1 osd.5 448 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:09.031 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:16:09.026+0000 16949700 -1 osd.3 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:45.315157+0000 front 2019-11-25T17:14:45.313288+0000 (oldest deadline 2019-11-25T17:15:29.626797+0000) 2019-11-25T17:16:09.227 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:16:09.221+0000 16949700 -1 osd.7 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.979097+0000 front 2019-11-25T17:14:48.663954+0000 (oldest deadline 2019-11-25T17:15:32.825719+0000) 2019-11-25T17:16:09.354 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:09.348+0000 16949700 -1 osd.1 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.632300+0000 front 2019-11-25T17:14:50.771166+0000 (oldest deadline 2019-11-25T17:15:30.539242+0000) 2019-11-25T17:16:09.484 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:16:09.469+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 45sec e448 v448) 2019-11-25T17:16:09.582 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:16:09.574+0000 16949700 -1 osd.0 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:50.771571+0000 front 2019-11-25T17:14:50.632277+0000 (oldest deadline 2019-11-25T17:15:32.845224+0000) 2019-11-25T17:16:09.628 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:09.621+0000 16949700 -1 osd.2 448 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:14:48.430676+0000 front 2019-11-25T17:14:48.347157+0000 (oldest deadline 2019-11-25T17:15:32.571995+0000) 2019-11-25T17:16:10.096 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:10.059+0000 16949700 -1 osd.5 449 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:11.169 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:11.117+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:12.115 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:12.110+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:13.102 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:13.097+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:14.142 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:14.137+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:15.151 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:15.146+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:16.120 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:16.115+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:17.150 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:17.145+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:18.200 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:18.194+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:19.213 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:19.209+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:20.194 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:20.188+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:21.159 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:21.154+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:22.129 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:22.124+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:23.172 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:23.167+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:24.153 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:24.148+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:25.188 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:25.183+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:26.214 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:26.209+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:27.194 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:27.188+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:28.171 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:28.165+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:29.161 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:29.156+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:30.168 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:30.163+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:31.157 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:31.152+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:32.199 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:32.195+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:33.194 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:16:33.190+0000 16949700 -1 osd.5 450 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122629 6.f 6:f6551702:root::%ef%84%91%ed%88%90%e1%a0%84%eb%8c%8e%ed%88%90%e1%a0%83%e1%a0%82%ef%84%8f%e5%98%85%e8%94%88%e9%90%8c%e5%98%86%e9%90%8b%ce%a9%ce%a9:head [create,call version.set in=58b,writefull 0~221 in=221b] snapc 0=[] ondisk+write+known_if_redirected e448) 2019-11-25T17:16:40.471 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:40.456+0000 16949700 -1 osd.6 451 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=44 ondisk+retry+read+known_if_redirected e449) 2019-11-25T17:16:41.651 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:41.647+0000 16949700 -1 osd.6 451 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=44 ondisk+retry+read+known_if_redirected e449) 2019-11-25T17:16:42.828 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:42.737+0000 16949700 -1 osd.6 451 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=44 ondisk+retry+read+known_if_redirected e449) 2019-11-25T17:16:42.979 INFO:teuthology.orchestra.run.smithi153.stderr:testCopy404s (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:16:44.099 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:44.086+0000 16949700 -1 osd.6 451 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=44 ondisk+retry+read+known_if_redirected e449) 2019-11-25T17:16:44.437 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:44.399+0000 34a6f700 -1 osd.2 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:44.523 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:16:44.485+0000 35270700 -1 osd.2 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:44.572 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:44.567+0000 3a27a700 -1 osd.6 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:44.611 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:44.602+0000 39278700 -1 osd.1 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:44.868 INFO:teuthology.orchestra.run.smithi153.stderr:testCopyDestinationSlashProblems (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:16:45.009 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:44.921+0000 3426e700 -1 osd.6 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:45.023 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:44.944+0000 37274700 -1 osd.1 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:45.067 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:44.945+0000 36272700 -1 osd.6 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:45.094 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:16:45.083+0000 37274700 -1 osd.1 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:45.262 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:45.149+0000 3b27c700 -1 osd.6 452 build_incremental_map_msg missing incremental map 452 2019-11-25T17:16:45.338 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:45.294+0000 16949700 -1 osd.6 452 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=44 ondisk+retry+read+known_if_redirected e449) 2019-11-25T17:16:46.646 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:16:46.485+0000 16949700 -1 osd.6 453 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=44 ondisk+retry+read+known_if_redirected e449) 2019-11-25T17:17:15.499 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:15.493+0000 16949700 -1 osd.4 454 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:16.579 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:16.553+0000 16949700 -1 osd.4 454 get_health_metrics reporting 6 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:16.706 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:16.702+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:17.536 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:17.531+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:17.661 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:17.657+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:18.573 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:18.568+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:18.639 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:18.635+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:19.620 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:19.615+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:19.641 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:19.637+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:20.615 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:20.609+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:20.690 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:20.686+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:21.640 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:21.634+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:21.696 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:21.692+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:22.679 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:22.675+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:22.713 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:22.709+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:23.641 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:23.637+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:23.698 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:23.693+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:24.678 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:24.672+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:24.691 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:24.687+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:25.640 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:25.636+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:25.707 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:25.703+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:26.627 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:26.623+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:26.745 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:26.741+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:27.600 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:27.595+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:27.756 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:27.752+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:28.561 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:28.556+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:28.784 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:28.778+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:29.609 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:29.605+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:29.832 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:29.828+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:30.654 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:30.649+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:30.802 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:30.798+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:31.655 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:31.650+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:31.789 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:31.785+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:32.639 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:32.634+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:32.745 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:32.741+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:33.654 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:33.650+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:33.796 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:33.792+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:34.635 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:34.630+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:34.832 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:34.828+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:35.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:35.599+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:35.846 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:35.841+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:35.847 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:35.842+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:36.564 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:36.559+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:36.853 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:36.849+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:36.855 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:36.851+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:37.160 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:37.154+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:37.569 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:37.563+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:37.617 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:37.611+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:37.875 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:37.869+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:37.875 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:37.870+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:38.016 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:38.011+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:38.177 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:38.169+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:38.569 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:38.563+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:38.570 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:38.564+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:38.610 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:38.605+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:38.917 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:38.912+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:38.918 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:38.913+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:39.042 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:39.037+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:39.130 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:39.125+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:39.229 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:39.225+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:39.527 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:39.521+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:39.530 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:39.525+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:39.566 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:39.562+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:39.915 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:02:44.497 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:39.924 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:02:44.507 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:39.929 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:02:44.534 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:39.931 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:39.927+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:39.933 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:02:44.538 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:39.933 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:39.929+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:39.934 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:02:44.540 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:39.936 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:02:44.542 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:40.053 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:40.048+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:40.106 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:40.101+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:40.175 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:02:44.780 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:40.184 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:40.179+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:40.185 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:02:44.790 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:40.224 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:02:44.806 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:17:40.533 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:40.528+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:40.551 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:40.543+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:40.552 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:40.545+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:40.770 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:40.764+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:40.892 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:40.887+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:40.892 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:40.888+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:41.051 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:41.046+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:41.116 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:41.111+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:41.237 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:41.232+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:41.541 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:41.536+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:41.567 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:41.561+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:41.568 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:41.562+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:41.756 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:41.749+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:41.921 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:41.916+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:41.922 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:41.917+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:42.032 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:42.027+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:42.145 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:42.136+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:42.276 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:42.271+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:42.562 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:42.557+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:42.615 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:42.608+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:42.615 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:42.610+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:42.714 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:42.707+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:42.900 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:42.895+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:42.901 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:42.896+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:43.039 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:43.031+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:43.164 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:43.159+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:43.261 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:43.256+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:43.551 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:43.546+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:43.648 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:43.642+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:43.651 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:43.646+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:43.694 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:43.688+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:43.890 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:43.885+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:43.892 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:43.888+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:44.068 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:44.063+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:44.144 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:44.139+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:44.249 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:44.244+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:44.519 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:44.514+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:44.631 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:44.625+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:44.632 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:44.627+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:44.705 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:44.698+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:44.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:44.939+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:44.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:44.940+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:45.123 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:45.119+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:45.168 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:45.162+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:45.264 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:45.259+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:45.475 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:45.471+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:45.638 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:45.632+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:45.638 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:45.633+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:45.687 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:45.680+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:45.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:45.940+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:45.947 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:45.944+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:46.168 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:46.163+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:46.220 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:46.215+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:46.307 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:46.303+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:46.491 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:46.486+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:46.596 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:46.588+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:46.597 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:46.592+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:46.662 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:46.656+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:46.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:46.914+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:46.921 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:46.917+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:47.145 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:47.140+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:47.181 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:47.176+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:47.327 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:47.323+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:47.514 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:47.510+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:47.629 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:47.622+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:47.630 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:47.624+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:47.680 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:47.674+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:47.885 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:47.880+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:47.886 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:47.881+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:48.125 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:48.120+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:48.187 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:48.182+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:48.350 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:48.345+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:48.571 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:48.564+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:48.614 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:48.608+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:48.615 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:48.610+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:48.707 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:48.701+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:48.908 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:48.903+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:48.908 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:48.904+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:49.142 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:49.137+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:49.172 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:49.166+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:49.373 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:49.369+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:49.576 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:49.571+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:49.601 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:49.595+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:49.601 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:49.595+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:49.693 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:49.688+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:49.870 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:49.866+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:49.873 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:49.869+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:50.136 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:50.128+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:50.200 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:50.195+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:50.408 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:50.403+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:50.558 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:50.553+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:50.657 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:50.652+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:50.660 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:50.655+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:50.708 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:50.702+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:50.864 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:50.859+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:50.866 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:50.862+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:51.152 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:51.148+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:51.178 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:51.172+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:51.365 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:51.360+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:51.552 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:51.547+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:51.682 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:51.677+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:51.690 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:51.683+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:51.691 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:51.685+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:51.856 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:51.850+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:51.856 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:51.852+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:52.112 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:52.107+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:52.217 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:52.212+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:52.355 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:52.351+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:52.562 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:52.556+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:52.648 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:52.641+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:52.648 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:52.643+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:52.653 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:52.648+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:52.907 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:52.903+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:52.910 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:52.906+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:53.148 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:53.143+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:53.185 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:53.179+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:53.356 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:53.351+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:53.533 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:53.527+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:53.618 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:53.612+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:53.657 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:53.650+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:53.658 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:53.652+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:53.900 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:53.894+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:53.900 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:53.895+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:54.112 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:54.107+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:54.146 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:54.141+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:54.386 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:54.381+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:54.535 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:54.530+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:54.624 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:54.618+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:54.625 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:54.618+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:54.626 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:54.620+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:54.926 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:54.921+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:54.927 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:54.923+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:55.105 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:55.101+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:55.157 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:55.151+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:55.377 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:55.372+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:55.559 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:55.554+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:55.619 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:55.614+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:55.620 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:55.615+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:55.669 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:55.663+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:55.939 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:55.934+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:55.940 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:55.935+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:56.141 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:56.136+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:56.156 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:56.150+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:56.393 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:56.388+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:56.600 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:56.594+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:56.604 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:56.599+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:56.606 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:56.601+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:56.645 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:56.638+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:56.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:56.960+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:56.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:56.961+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:57.134 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:57.129+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:57.173 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:57.167+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:57.361 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:57.357+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:57.561 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:57.556+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:57.640 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:57.634+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:57.657 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:57.652+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:57.659 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:57.654+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:57.952 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:57.946+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:57.952 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:57.948+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:58.143 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:58.137+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:58.176 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:58.170+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:58.344 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:58.339+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:58.561 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:58.556+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:58.603 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:58.597+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:17:58.617 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:58.609+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:58.617 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:58.612+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:58.961 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:58.956+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:17:58.964 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:58.960+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:17:59.170 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:17:59.163+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:17:59.191 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:17:59.186+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:17:59.324 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:17:59.319+0000 16949700 -1 osd.0 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.073753+0000 front 2019-11-25T17:16:53.187577+0000 (oldest deadline 2019-11-25T17:17:38.873797+0000) 2019-11-25T17:17:59.572 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:17:59.567+0000 16949700 -1 osd.1 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.928399+0000 front 2019-11-25T17:16:55.175688+0000 (oldest deadline 2019-11-25T17:17:37.193498+0000) 2019-11-25T17:17:59.612 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:59.607+0000 16949700 -1 osd.4 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.701161+0000 front 2019-11-25T17:16:54.334435+0000 (oldest deadline 2019-11-25T17:17:38.337687+0000) 2019-11-25T17:17:59.615 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:17:59.610+0000 16949700 -1 osd.4 454 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e452) 2019-11-25T17:17:59.649 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:17:59.643+0000 16949700 -1 osd.5 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.586625+0000 front 2019-11-25T17:16:54.571172+0000 (oldest deadline 2019-11-25T17:17:39.832505+0000) 2019-11-25T17:18:00.000 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:59.991+0000 16949700 -1 osd.3 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:54.128590+0000 front 2019-11-25T17:16:54.107204+0000 (oldest deadline 2019-11-25T17:17:35.147116+0000) 2019-11-25T17:18:00.023 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:17:59.997+0000 16949700 -1 osd.3 454 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:18:00.123 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:18:00.118+0000 16949700 -1 osd.2 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:53.969025+0000 front 2019-11-25T17:16:53.570391+0000 (oldest deadline 2019-11-25T17:17:37.558632+0000) 2019-11-25T17:18:00.163 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:18:00.158+0000 16949700 -1 osd.7 454 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:16:51.187988+0000 front 2019-11-25T17:16:51.275283+0000 (oldest deadline 2019-11-25T17:17:36.451413+0000) 2019-11-25T17:18:01.003 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:18:00.999+0000 16949700 -1 osd.3 455 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:18:02.184 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:18:02.176+0000 16949700 -1 osd.3 456 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:122829 2.2s0 2:42aa2d45:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e3%9c%83%ef%84%92%e8%94%8b%e1%a0%82%e1%a0%81%e8%94%8a%e1%a0%82%e3%9c%83%e9%90%8b%e3%9c%84%ef%84%90%ce%a9%eb%8c%8c%e0%a4%82%ef%84%90:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~2571 in=2571b,setxattr user.rgw.manifest (384) in=401b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,setxattr user.rgw.x-amz-meta-371c855604b04cb5988397cdd3aafcb1 (46) in=98b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e452) 2019-11-25T17:18:24.671 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:18:24.665+0000 38a77700 -1 osd.3 456 build_incremental_map_msg missing incremental map 442 2019-11-25T17:18:45.130 INFO:teuthology.orchestra.run.smithi153.stderr:testCopyFromHeader (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:18:47.371 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:18:47.345+0000 34a6f700 -1 osd.6 454 build_incremental_map_msg missing incremental map 452 2019-11-25T17:18:50.183 INFO:teuthology.orchestra.run.smithi153.stderr:testCopyFromHeader404s (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:18:51.416 INFO:teuthology.orchestra.run.smithi153.stderr:testCopyNoDestinationHeader (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:19:03.967 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:03.963+0000 38a77700 -1 osd.6 459 build_incremental_map_msg missing incremental map 428 2019-11-25T17:19:03.976 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:03.972+0000 38a77700 -1 osd.6 459 build_incremental_map_msg missing incremental map 442 2019-11-25T17:19:03.984 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:03.979+0000 38a77700 -1 osd.6 459 build_incremental_map_msg missing incremental map 448 2019-11-25T17:19:03.985 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:03.980+0000 38a77700 -1 osd.6 459 build_incremental_map_msg missing incremental map 452 2019-11-25T17:19:07.801 INFO:teuthology.orchestra.run.smithi153.stderr:testDelete (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:19:08.939 INFO:teuthology.orchestra.run.smithi153.stderr:testDeleteOfFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:19:16.193 INFO:teuthology.orchestra.run.smithi153.stderr:testDeleteThen404s (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:19:19.633 INFO:teuthology.orchestra.run.smithi153.stderr:testEtagResponse (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:19:19.693 INFO:teuthology.orchestra.run.smithi153.stderr:testEtagWayoff (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:19:50.237 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:50.229+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:50.890 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:50.885+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:51.174 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:51.168+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:51.213 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:51.207+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:51.595 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:51.590+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:51.851 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:51.845+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:52.174 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:52.169+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:52.178 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:52.171+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:52.563 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:52.557+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:52.653 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:19:52.646+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:19:52.862 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:52.857+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:53.128 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:19:53.123+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:19:53.148 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:53.142+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:19:53.148 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:53.143+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:53.182 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:53.176+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:53.578 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:53.572+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:53.667 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:19:53.661+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:19:53.898 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:53.893+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:54.105 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:19:54.100+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:19:54.135 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:54.129+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:54.169 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:54.162+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:19:54.170 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:54.164+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:54.577 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:54.572+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:54.716 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:19:54.708+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:19:54.908 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:54.902+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:55.145 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:55.138+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:55.148 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:19:55.142+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:19:55.188 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:55.182+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:19:55.189 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:55.184+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:55.556 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:55.551+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:55.676 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:19:55.668+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:19:55.882 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:55.877+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:56.164 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:19:56.159+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:19:56.167 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:56.162+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:19:56.168 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:56.163+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:56.171 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:56.166+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:56.511 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:56.506+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:56.725 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:19:56.718+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:19:56.868 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:56.864+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:57.136 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:19:57.130+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:19:57.163 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:57.157+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:57.214 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:57.209+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:19:57.215 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:57.211+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:57.501 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:57.496+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:57.680 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:19:57.676+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:19:57.887 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:57.881+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:58.114 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:19:58.107+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:19:58.201 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:58.195+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:58.236 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:58.231+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:19:58.237 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:58.232+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:58.498 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:19:58.494+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:19:58.551 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:58.546+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:58.724 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:19:58.719+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:19:58.923 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:58.918+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:19:59.159 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:19:59.154+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:19:59.160 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:19:59.154+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:19:59.231 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:59.226+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:19:59.233 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:19:59.229+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:19:59.552 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:19:59.547+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:19:59.596 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:19:59.591+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:19:59.702 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:19:59.697+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:19:59.912 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:19:59.907+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:00.145 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:00.139+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:00.148 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:00.140+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:00.186 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:00.181+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:00.188 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:00.184+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:00.566 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:00.560+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:00.574 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:00.569+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:00.685 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:00.679+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:00.892 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:00.886+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:01.106 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:01.101+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:01.134 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:01.127+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:01.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:01.197+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:01.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:01.199+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:01.559 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:01.554+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:01.626 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:01.621+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:01.739 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:01.733+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:01.863 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:01.857+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:02.077 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:02.072+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:02.119 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:02.113+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:02.199 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:02.194+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:02.202 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:02.198+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:02.538 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:02.532+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:02.620 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:02.614+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:02.727 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:02.723+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:02.903 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:02.898+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:03.103 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:03.098+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:03.114 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:03.109+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:03.201 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:03.195+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:03.201 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:03.197+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:03.557 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:03.552+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:03.579 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:03.574+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:03.721 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:03.715+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:03.878 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:03.874+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:04.107 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:04.102+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:04.132 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:04.123+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:04.177 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:04.171+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:04.178 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:04.173+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:04.536 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:04.530+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:04.549 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:04.544+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:04.728 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:04.723+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:04.838 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:04.833+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:05.147 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:05.142+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:05.152 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:05.148+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:05.154 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:05.150+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:05.155 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:05.150+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:05.493 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:05.487+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:05.536 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:05.530+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:05.693 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:05.688+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:05.793 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:05.788+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:06.104 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:06.099+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:06.146 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:06.142+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:06.152 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:06.147+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:06.152 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:06.148+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:06.516 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:06.511+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:06.532 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:06.527+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:06.702 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:06.696+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:06.766 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:06.761+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:07.076 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:07.071+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:07.133 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:07.129+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:07.135 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:07.131+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:07.194 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:07.190+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:07.504 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:07.494+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:07.508 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:07.502+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:07.691 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:07.682+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:07.753 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:07.746+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:08.101 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:08.096+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:08.112 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:08.107+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:08.113 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:08.108+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:08.217 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:08.212+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:08.494 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:08.488+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:08.556 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:08.550+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:08.655 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:08.649+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:08.759 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:08.753+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:09.094 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:09.086+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:09.112 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:09.107+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:09.113 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:09.109+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:09.270 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:09.265+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:09.500 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:09.493+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:09.560 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:09.554+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:09.683 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:09.678+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:09.728 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:09.722+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:10.077 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:10.072+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:10.084 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:10.078+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:10.085 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:10.080+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:10.232 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:10.226+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:10.472 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:10.467+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:10.517 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:10.512+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:10.710 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:10.703+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:10.745 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:10.738+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:11.063 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:11.058+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:11.066 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:11.062+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:11.076 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:11.071+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:11.236 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:11.230+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:11.501 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:11.496+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:11.516 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:11.510+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:11.716 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:11.711+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:11.731 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:11.724+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:12.064 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:12.058+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:12.097 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:12.092+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:12.099 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:12.095+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:12.261 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:12.256+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:12.506 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:12.500+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:12.524 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:12.518+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:12.727 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:12.722+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:12.767 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:12.762+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:13.053 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:13.047+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:13.053 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:13.049+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:13.086 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:13.081+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:13.308 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:13.304+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:13.494 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:13.488+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:13.545 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:13.539+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:13.742 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:13.738+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:13.758 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:13.752+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:14.062 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:14.056+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:14.062 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:14.057+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:14.125 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:14.119+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:14.327 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:14.322+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:14.531 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:14.526+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:14.547 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:14.541+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:14.784 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:14.778+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:14.787 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:14.782+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:15.056 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:15.047+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:15.057 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:15.050+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:15.177 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:15.172+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:15.374 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:15.369+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:15.524 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:15.518+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:15.541 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:15.535+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:15.821 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:15.815+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:15.831 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:15.826+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:16.051 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:16.046+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:16.054 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:16.050+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:16.204 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:16.199+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:16.355 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:16.349+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:16.478 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:16.473+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:16.583 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:16.577+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:16.808 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:16.803+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:16.824 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:16.819+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:17.085 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:17.080+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:17.085 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:17.081+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:17.215 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:17.211+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:17.329 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:17.323+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:17.484 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:17.477+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:17.577 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:17.571+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:17.789 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:17.783+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:17.868 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:17.861+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:18.081 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:18.076+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:18.084 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:18.080+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:18.170 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:18.166+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:18.323 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:18.318+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:18.478 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:18.471+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:18.588 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:18.582+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:18.780 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:18.774+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:18.880 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:18.874+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:19.056 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:19.051+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:19.059 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:19.055+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:19.132 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:19.127+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:19.338 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:19.332+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:19.464 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:19.458+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:19.641 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:19.634+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:19.786 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:19.780+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:19.895 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:19.890+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:20.019 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:20.013+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:20.020 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:20.015+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:20.117 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:20.112+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:20.296 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:20.291+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:20.387 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:20:20.381+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 41sec e459 v459) 2019-11-25T17:20:20.430 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:20.423+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:20.601 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:20.595+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:20.796 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:20.791+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:20.872 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:20.861+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:20.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:20.987+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:20.993 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:20.988+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:21.157 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:21.151+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:21.350 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:21.343+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:21.421 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:21.415+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:21.557 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:21.550+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:21.773 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:21.767+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:21.862 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:21.857+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:21.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:21.986+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:21.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:21.988+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:22.194 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:22.188+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:22.316 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:22.310+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:22.444 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:22.437+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:22.523 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:22.514+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:22.811 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:22.805+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:22.886 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:22.879+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:22.991 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:22.984+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:22.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:22.987+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:23.229 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:23.225+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:23.370 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:23.361+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:23.419 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:23.413+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:23.485 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:23.479+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:23.822 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:23.817+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:23.885 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:23.879+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:24.014 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:24.009+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:24.017 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:24.013+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:24.214 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:24.208+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:24.345 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:24.340+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:24.379 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:24.372+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:24.499 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:24.493+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:24.829 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:24.823+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:24.917 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:24.912+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:25.009 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:25.004+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:25.011 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:25.007+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:25.266 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:25.261+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:25.336 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:25.331+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:25.376 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:25.370+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:25.406 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:20:25.400+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 41sec e459 v459) 2019-11-25T17:20:25.538 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:25.532+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:25.877 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:25.872+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:25.967 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:25.961+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:25.980 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:25.974+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:25.981 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:25.976+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:26.241 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:26.235+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:26.312 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:26.306+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:26.394 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:26.388+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:26.570 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:26.564+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:26.833 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:26.828+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:26.998 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:26.991+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:27.007 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:27.002+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:27.009 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:27.004+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:27.265 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:27.260+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:27.283 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:27.278+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:27.436 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:27.429+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:27.618 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:27.612+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:27.842 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:27.837+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:27.985 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:27.980+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:27.988 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:27.984+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:28.029 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:28.023+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:28.241 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:28.237+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:28.291 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:28.285+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:28.438 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:28.431+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:28.624 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:28.617+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:28.895 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:28.890+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:28.972 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:28.966+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:28.974 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:28.967+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:29.058 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:29.053+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:29.241 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:29.235+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:29.305 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:29.299+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:29.481 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:29.475+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:29.640 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:29.635+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:29.912 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:29.907+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:30.008 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:30.002+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:30.009 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:30.004+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:30.084 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:30.077+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:30.236 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:30.231+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:30.265 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:30.260+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:30.433 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:20:30.423+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 41sec e459 v459) 2019-11-25T17:20:30.498 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:30.493+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:30.640 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:30.634+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:30.962 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:30.957+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:30.993 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:30.988+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:30.994 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:30.989+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:31.052 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:31.045+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:31.221 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:31.216+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:31.277 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:31.272+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:31.538 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:31.533+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:31.605 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:31.600+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:32.006 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:32.000+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:32.026 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:32.021+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:32.027 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:32.023+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:32.063 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:32.057+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:32.224 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:32.214+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:32.232 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:32.228+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:32.525 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:32.518+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:32.614 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:32.608+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:33.013 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:33.008+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:33.028 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:33.022+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:33.029 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:33.025+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:33.103 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:33.098+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:33.237 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:33.232+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:33.282 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:33.276+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:33.534 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:33.528+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:33.656 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:33.650+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:34.023 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:34.016+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:34.062 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:34.056+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:34.069 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:34.063+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:34.087 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:34.081+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:34.234 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:34.229+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:34.316 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:34.311+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:34.537 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:34.531+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:34.708 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:34.702+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:35.034 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:35.028+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:35.111 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:35.107+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:35.113 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:35.110+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:35.115 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:35.109+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:35.273 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:35.268+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:35.342 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:35.337+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:35.448 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:20:35.443+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 41sec e459 v459) 2019-11-25T17:20:35.565 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:35.559+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:35.734 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:35.727+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:36.060 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:36.055+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:36.093 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:36.088+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:36.094 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:36.089+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:36.162 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:36.157+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:36.310 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:36.304+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:36.321 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:36.316+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:36.534 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:36.528+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:36.740 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:36.734+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:37.056 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:37.050+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:37.124 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:37.119+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:37.125 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:37.120+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:37.136 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:37.129+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:37.278 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:37.274+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:37.357 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:37.351+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:37.519 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:37.513+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:37.790 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:37.784+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:38.083 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:38.078+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:38.112 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:38.106+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:38.134 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:38.130+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:38.136 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:38.132+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:38.329 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:38.323+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:38.348 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:38.342+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:38.532 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:38.526+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:38.768 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:38.761+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:39.096 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:39.089+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:39.096 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:39.091+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:39.125 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:39.120+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:39.127 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:39.122+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:39.308 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:39.302+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:39.374 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:39.370+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:39.511 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:39.505+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:39.735 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:20:39.729+0000 16949700 -1 osd.6 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:09.578616+0000 front 2019-11-25T17:19:13.706177+0000 (oldest deadline 2019-11-25T17:19:50.722604+0000) 2019-11-25T17:20:40.054 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:40.050+0000 16949700 -1 osd.3 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.354383+0000 front 2019-11-25T17:19:12.682046+0000 (oldest deadline 2019-11-25T17:19:52.463913+0000) 2019-11-25T17:20:40.056 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:40.052+0000 16949700 -1 osd.3 459 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:40.133 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:20:40.127+0000 16949700 -1 osd.5 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.465526+0000 front 2019-11-25T17:19:16.325111+0000 (oldest deadline 2019-11-25T17:19:50.653982+0000) 2019-11-25T17:20:40.138 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:20:40.133+0000 16949700 -1 osd.1 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:12.698021+0000 front 2019-11-25T17:19:08.356383+0000 (oldest deadline 2019-11-25T17:19:52.630791+0000) 2019-11-25T17:20:40.263 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:20:40.258+0000 16949700 -1 osd.0 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:08.440945+0000 front 2019-11-25T17:19:08.447307+0000 (oldest deadline 2019-11-25T17:19:52.607382+0000) 2019-11-25T17:20:40.399 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:20:40.393+0000 16949700 -1 osd.2 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:16.308128+0000 front 2019-11-25T17:19:08.249900+0000 (oldest deadline 2019-11-25T17:19:50.219424+0000) 2019-11-25T17:20:40.543 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:20:40.538+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 41sec e459 v459) 2019-11-25T17:20:40.545 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:20:40.539+0000 16949700 -1 osd.7 459 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:19:19.966311+0000 front 2019-11-25T17:19:20.028554+0000 (oldest deadline 2019-11-25T17:19:58.052894+0000) 2019-11-25T17:20:41.434 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:41.362+0000 16949700 -1 osd.3 460 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:42.529 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:42.468+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:43.517 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:43.511+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:44.485 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:44.480+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:45.465 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:45.459+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:46.459 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:46.455+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:47.505 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:47.499+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:48.533 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:48.528+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:49.549 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:49.545+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:50.565 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:50.560+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:51.519 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:51.515+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:52.523 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:52.519+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:53.487 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:53.482+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:54.539 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:54.535+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:55.574 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:55.570+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:56.554 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:56.550+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:57.571 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:57.567+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:58.571 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:58.566+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:20:59.610 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:20:59.606+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:21:00.617 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:00.607+0000 16949700 -1 osd.3 461 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123208 2.5s0 2:a1065017:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%ef%84%90%e8%94%89%ed%88%8e%e1%a0%84%ed%88%90%eb%8c%8b%e5%98%86%e5%98%87%e3%9c%84%eb%8c%8b%ef%84%90%e8%94%89%e1%a0%84%e8%94%8b%e5%98%88:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~29043 in=29043b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (25) in=46b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e459) 2019-11-25T17:21:04.182 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:04.138+0000 3426e700 -1 osd.1 461 build_incremental_map_msg missing incremental map 432 2019-11-25T17:21:04.192 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:04.150+0000 3426e700 -1 osd.1 461 build_incremental_map_msg missing incremental map 452 2019-11-25T17:21:11.092 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:11.064+0000 38a77700 -1 osd.4 464 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:11.260 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:11.247+0000 3aa7b700 -1 osd.4 464 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:11.276 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:11.259+0000 37274700 -1 osd.4 464 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:11.277 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:11.263+0000 124aa700 -1 osd.4 464 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:11.308 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:11.284+0000 33a6d700 -1 osd.4 464 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:11.310 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:11.292+0000 38276700 -1 osd.4 464 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:11.748 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:11.520+0000 38a77700 -1 osd.4 464 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:21.027 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:20.936+0000 33a6d700 -1 osd.4 465 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:25.818 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:06:30.401 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:25.853 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:06:30.459 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:25.868 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:06:30.450 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:25.880 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:06:30.485 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:25.882 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:06:30.488 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:25.890 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:06:30.496 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:26.075 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:06:30.681 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:26.110 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:06:30.693 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:26.158 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:06:30.764 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.169 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:06:32.751 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.176 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:06:32.758 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.186 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:06:32.792 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.191 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:06:32.797 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.196 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:06:32.801 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.199 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:06:32.805 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.211 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:06:32.793 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.219 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:06:32.824 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:28.235 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:06:32.841 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:21:32.832 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:32.827+0000 33a6d700 -1 osd.4 465 build_incremental_map_msg missing incremental map 464 2019-11-25T17:21:37.683 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:37.675+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:38.725 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:38.718+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:39.724 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:39.718+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:39.791 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:39.785+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:39.904 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:39.897+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:40.720 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:40.714+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:40.752 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:40.748+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:40.834 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:40.828+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:40.946 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:40.940+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:41.203 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:41.197+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:41.751 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:41.744+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:41.790 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:41.786+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:41.791 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:41.785+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:41.837 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:41.832+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:41.934 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:41.927+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:42.218 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:42.211+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:42.521 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:42.516+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:42.747 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:42.743+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:42.793 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:42.788+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:42.834 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:42.824+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:42.888 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:42.884+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:42.923 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:42.918+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:43.190 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:43.186+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:43.520 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:43.509+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:43.730 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:43.723+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:43.818 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:43.812+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:43.881 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:43.877+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:43.909 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:43.904+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:43.973 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:43.966+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:44.219 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:44.215+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:44.556 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:44.549+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:44.755 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:44.750+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:44.853 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:44.848+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:44.855 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:44.849+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:44.956 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:44.952+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:44.983 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:44.978+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:45.234 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:45.229+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:45.559 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:45.554+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:45.792 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:45.787+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:45.828 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:45.823+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:45.872 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:45.866+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:45.927 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:45.923+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:45.981 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:45.977+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:46.190 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:46.185+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:46.586 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:46.581+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:46.770 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:46.765+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:46.839 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:46.833+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:46.868 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:46.863+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:46.956 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:46.953+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:47.015 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:47.007+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:47.238 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:47.234+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:47.606 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:47.601+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:47.787 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:47.781+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:47.866 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:47.861+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:47.882 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:47.877+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:48.005 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:48.001+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:48.009 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:48.005+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:48.060 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:48.054+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:48.223 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:48.218+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:48.575 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:48.570+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:48.769 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:48.765+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:48.819 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:48.814+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:48.898 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:48.893+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:49.016 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:49.011+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:49.017 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:49.013+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:49.083 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:49.077+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:49.255 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:49.249+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:49.552 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:49.547+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:49.790 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:49.784+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:49.804 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:49.799+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:49.948 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:49.944+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:50.048 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:50.042+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:50.049 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:50.044+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:50.115 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:50.109+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:50.257 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:50.252+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:50.602 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:50.595+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:50.746 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:50.741+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:50.793 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:50.788+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:50.911 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:50.906+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:51.014 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:51.007+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:51.016 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:51.009+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:51.120 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:51.116+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:51.301 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:51.296+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:51.578 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:51.572+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:51.778 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:51.772+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:51.780 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:51.775+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:51.877 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:51.872+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:52.051 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:52.045+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:52.055 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:52.050+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:52.092 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:52.087+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:52.349 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:52.344+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:52.610 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:52.605+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:52.788 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:52.783+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:52.799 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:52.794+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:52.895 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:52.891+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:53.023 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:53.019+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:53.025 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:53.021+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:53.056 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:53.052+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:53.344 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:53.339+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:53.635 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:53.629+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:53.794 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:53.788+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:53.847 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:53.842+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:53.918 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:53.913+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:54.056 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:54.049+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:54.057 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:54.051+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:54.063 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:54.058+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:54.324 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:54.319+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:54.686 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:54.680+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:54.761 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:54.756+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:54.876 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:54.870+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:54.914 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:54.909+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:55.052 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:55.047+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:55.073 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:55.068+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:55.074 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:55.069+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:55.323 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:55.318+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:55.698 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:55.692+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:55.795 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:55.790+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:55.903 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:55.897+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:55.951 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:55.945+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:56.092 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:56.087+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:56.120 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:56.114+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:56.121 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:56.116+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:56.312 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:56.306+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:56.679 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:56.673+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:56.785 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:56.780+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:56.945 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:56.935+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:56.970 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:56.965+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:57.072 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:57.067+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:57.141 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:57.136+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:57.142 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:57.137+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:57.337 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:57.332+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:57.692 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:57.686+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:57.806 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:57.801+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:57.903 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:57.898+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:57.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:57.986+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:58.126 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:58.120+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:58.187 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:58.181+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:58.188 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:58.182+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:58.379 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:58.374+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:58.724 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:58.719+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:58.765 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:58.760+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:58.884 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:58.879+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:21:59.034 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:21:59.029+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:21:59.097 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:21:59.092+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:21:59.156 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:59.151+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:21:59.157 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:21:59.152+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:21:59.400 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:21:59.395+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:21:59.695 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:21:59.689+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:21:59.762 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:21:59.757+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:21:59.877 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:21:59.872+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:00.037 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:00.032+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:00.112 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:00.107+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:00.161 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:00.155+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:00.162 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:00.157+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:00.415 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:00.408+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:00.698 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:00.691+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:00.791 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:00.786+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:00.904 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:00.898+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:00.993 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:00.988+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:01.130 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:01.125+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:01.211 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:01.205+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:01.212 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:01.207+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:01.461 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:01.456+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:01.714 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:01.710+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:01.753 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:01.749+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:01.921 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:01.915+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:01.955 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:01.950+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:02.163 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:02.157+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:02.223 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:02.218+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:02.226 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:02.221+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:02.475 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:02.471+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:02.693 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:02.687+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:02.734 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:02.729+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:02.886 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:02.880+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:02.984 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:02.978+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:03.160 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:03.156+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:03.193 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:03.187+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:03.193 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:03.188+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:03.445 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:03.440+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:03.688 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:03.682+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:03.719 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:03.713+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:03.869 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:03.863+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:04.018 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:04.012+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:04.188 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:04.182+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:04.231 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:04.225+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:04.232 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:04.226+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:04.411 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:04.407+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:04.673 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:04.667+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:04.751 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:04.746+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:04.917 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:04.911+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:04.984 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:04.978+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:05.183 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:05.179+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:05.196 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:05.189+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:05.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:05.191+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:05.441 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:05.437+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:05.644 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:05.637+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:05.759 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:05.754+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:05.877 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:05.872+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:05.962 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:05.958+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:06.170 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:06.163+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:06.170 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:06.165+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:06.203 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:06.198+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:06.443 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:06.438+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:06.646 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:06.640+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:06.740 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:06.735+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:06.866 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:06.860+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:06.924 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:06.918+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:07.190 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:07.186+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:07.208 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:07.202+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:07.209 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:07.204+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:07.481 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:07.477+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:07.687 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:07.681+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:07.748 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:07.742+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:07.901 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:07.895+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:07.907 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:07.902+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:08.196 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:08.191+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:08.196 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:08.190+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:08.199 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:08.194+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:08.524 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:08.519+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:08.680 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:08.674+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:08.741 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:08.736+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:08.951 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:08.945+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:08.960 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:08.954+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:09.153 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:09.147+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:09.153 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:09.148+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:09.194 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:09.189+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:09.556 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:09.551+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:09.670 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:09.664+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:09.704 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:09.699+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:09.929 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:09.923+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:09.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:09.986+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:10.162 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:10.155+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:10.162 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:10.157+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:10.206 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:10.197+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:10.535 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:10.529+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:10.631 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:10.625+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:10.694 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:10.685+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:10.901 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:10.896+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:11.024 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:11.019+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:11.030 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:22:11.025+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e465 v465) 2019-11-25T17:22:11.169 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:11.163+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:11.170 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:11.165+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:11.220 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:11.215+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:11.544 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:11.540+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:11.599 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:11.594+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:11.702 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:11.697+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:11.918 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:11.912+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:12.075 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:12.069+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:12.175 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:12.168+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:12.176 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:12.170+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:12.197 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:12.191+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:12.558 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:12.552+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:12.585 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:12.580+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:12.708 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:12.704+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:12.872 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:12.867+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:13.127 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:13.122+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:13.156 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:13.151+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:13.159 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:13.154+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:13.250 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:13.245+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:13.551 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:13.546+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:13.570 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:13.564+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:13.745 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:13.740+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:13.829 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:13.824+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:14.113 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:14.108+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:14.163 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:14.158+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:14.167 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:14.161+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:14.235 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:14.228+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:14.511 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:14.507+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:14.527 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:14.522+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:14.720 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:14.714+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:14.862 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:14.856+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:15.130 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:15.125+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:15.212 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:15.207+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:15.213 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:15.209+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:15.256 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:15.252+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:15.530 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:15.523+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:15.545 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:15.539+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:15.728 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:15.724+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:15.874 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:15.868+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:16.050 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:22:16.043+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e465 v465) 2019-11-25T17:22:16.159 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:16.153+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:16.196 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:16.189+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:16.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:16.191+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:16.224 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:16.219+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:16.538 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:16.533+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:16.584 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:16.579+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:16.728 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:16.722+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:16.862 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:16.856+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:17.192 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:17.186+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:17.195 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:17.190+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:17.201 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:17.195+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:17.219 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:17.211+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:17.550 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:17.545+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:17.590 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:17.585+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:17.683 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:17.678+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:17.913 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:17.908+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:18.163 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:18.158+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:18.166 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:18.161+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:18.195 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:18.190+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:18.202 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:18.197+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:18.572 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:18.567+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:18.610 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:18.604+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:18.649 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:18.644+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:18.958 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:18.952+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:19.197 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:19.192+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:19.199 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:19.194+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:19.227 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:19.222+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:19.245 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:19.240+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:19.534 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:19.530+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:19.629 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:19.623+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:19.652 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:19.646+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:19.952 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:19.946+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:20.163 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:20.156+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:20.164 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:20.158+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:20.237 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:20.232+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:20.261 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:20.256+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:20.554 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:22:20.549+0000 16949700 -1 osd.0 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.766651+0000 front 2019-11-25T17:20:57.867184+0000 (oldest deadline 2019-11-25T17:21:40.496617+0000) 2019-11-25T17:22:20.615 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:22:20.611+0000 16949700 -1 osd.1 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.986666+0000 front 2019-11-25T17:20:58.095345+0000 (oldest deadline 2019-11-25T17:21:37.659079+0000) 2019-11-25T17:22:20.650 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:22:20.644+0000 16949700 -1 osd.5 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:58.095407+0000 front 2019-11-25T17:20:58.096095+0000 (oldest deadline 2019-11-25T17:21:41.971843+0000) 2019-11-25T17:22:20.932 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:22:20.926+0000 16949700 -1 osd.7 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:55.035316+0000 front 2019-11-25T17:20:54.966208+0000 (oldest deadline 2019-11-25T17:21:40.283657+0000) 2019-11-25T17:22:21.123 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:22:21.113+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 6 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 42sec e465 v465) 2019-11-25T17:22:21.130 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:21.125+0000 16949700 -1 osd.4 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 ever on either front or back, first ping sent 2019-11-25T17:21:07.905553+0000 (oldest deadline 2019-11-25T17:21:47.905553+0000) 2019-11-25T17:22:21.132 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:21.127+0000 16949700 -1 osd.4 465 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e463) 2019-11-25T17:22:21.202 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:22:21.197+0000 16949700 -1 osd.3 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:54.776528+0000 front 2019-11-25T17:20:54.778667+0000 (oldest deadline 2019-11-25T17:21:38.910227+0000) 2019-11-25T17:22:21.275 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:22:21.270+0000 16949700 -1 osd.2 465 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:20:57.681010+0000 front 2019-11-25T17:20:58.071195+0000 (oldest deadline 2019-11-25T17:21:38.997430+0000) 2019-11-25T17:22:21.673 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:22:21.644+0000 34a6f700 -1 osd.4 466 build_incremental_map_msg missing incremental map 464 2019-11-25T17:22:54.332 INFO:teuthology.orchestra.run.smithi153.stderr:testFileCreate (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:23:26.370 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:23:26.365+0000 16949700 -1 osd.6 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123627 2.3ds0 2:bc9219a9:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e9%90%8a%e8%94%88%e9%90%8c%e5%98%87%ef%84%92%e9%90%8c%e0%a4%83%e8%94%88%ef%84%8f%e1%a0%81%e2%98%83%ed%88%8e%e9%90%8c%e5%98%85%e9%90%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~15396 in=15396b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (45) in=66b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e469) 2019-11-25T17:23:27.333 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:23:27.329+0000 16949700 -1 osd.6 471 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:123627 2.3ds0 2:bc9219a9:::103c5a94-eb69-4977-8c90-9f6d634d8761.4358.658_%e9%90%8a%e8%94%88%e9%90%8c%e5%98%87%ef%84%92%e9%90%8c%e0%a4%83%e8%94%88%ef%84%8f%e1%a0%81%e2%98%83%ed%88%8e%e9%90%8c%e5%98%85%e9%90%8b:head [create,setxattr user.rgw.idtag (48) in=62b,setxattr user.rgw.tail_tag (48) in=65b,writefull 0~15396 in=15396b,setxattr user.rgw.manifest (385) in=402b,setxattr user.rgw.acl (193) in=205b,setxattr user.rgw.compression (56) in=76b,setxattr user.rgw.content_type (45) in=66b,setxattr user.rgw.etag (32) in=45b,call rgw.obj_store_pg_ver in=44b,setxattr user.rgw.source_zone (4) in=24b] snapc 0=[] ondisk+write+known_if_redirected e469) 2019-11-25T17:23:29.413 INFO:teuthology.orchestra.run.smithi153.stderr:testGetContentType (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:23:30.307 INFO:teuthology.orchestra.run.smithi153.stderr:testGetOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:23:34.306 INFO:teuthology.orchestra.run.smithi153.stderr:testHead (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:23:34.762 INFO:teuthology.orchestra.run.smithi153.stderr:testHeadOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:24:01.101 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:01.095+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:01.191 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:01.186+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:02.066 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:02.060+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:02.203 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:02.197+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:02.337 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:02.332+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:02.885 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:02.880+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:03.078 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:03.071+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:03.194 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:03.189+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:03.299 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:03.293+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:03.911 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:03.905+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:04.130 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:04.125+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:04.214 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:04.209+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:04.219 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:04.212+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:04.316 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:04.310+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:04.421 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:04.406+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:04.918 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:04.911+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:05.181 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:05.175+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:05.228 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:05.223+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:05.266 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:05.260+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:05.303 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:05.296+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:05.309 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:05.302+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:05.464 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:05.458+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:05.939 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:05.933+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:06.229 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:06.223+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:06.244 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:06.239+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:06.249 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:06.244+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:06.291 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:06.286+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:06.335 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:06.328+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:06.462 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:06.456+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:06.995 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:06.988+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:07.196 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:07.189+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:07.272 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:07.266+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:07.298 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:07.293+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:07.333 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:07.327+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:07.341 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:07.335+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:07.432 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:07.426+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:07.977 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:07.971+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:08.189 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:08.183+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:08.230 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:08.225+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:08.298 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:08.293+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:08.350 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:08.345+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:08.385 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:08.379+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:08.409 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:08.404+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:08.960 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:08.953+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:09.152 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:09.146+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:09.242 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:09.233+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:09.328 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:09.322+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:09.398 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:09.390+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:09.404 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:09.399+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:09.417 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:09.411+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:09.922 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:09.917+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:10.200 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:10.194+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:10.244 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:10.238+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:10.346 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:10.340+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:10.412 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:10.406+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:10.419 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:10.414+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:10.430 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:10.426+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:10.916 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:10.911+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:11.256 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:11.249+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:11.353 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:11.348+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:11.417 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:11.413+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:11.418 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:11.412+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:11.434 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:11.430+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:11.465 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:11.460+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:11.877 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:09:16.459 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:11.918 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:09:16.500 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:11.927 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:11.920+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:11.947 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:09:16.552 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:11.981 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:09:16.586 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:11.983 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:09:16.588 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:11.987 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:09:16.592 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:12.092 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:09:16.674 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:12.151 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:09:16.756 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:12.281 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:09:16.887 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:24:12.298 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:12.294+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:12.376 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:12.372+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:12.387 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:12.383+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:12.452 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:12.448+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:12.466 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:12.460+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:12.548 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:12.542+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:12.946 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:12.941+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:13.334 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:13.329+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:13.414 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:13.409+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:13.437 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:13.430+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:13.450 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:13.445+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:13.469 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:13.465+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:13.517 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:13.513+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:13.955 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:13.949+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:14.341 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:14.336+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:14.401 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:14.394+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:14.408 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:14.404+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:14.433 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:14.428+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:14.448 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:14.443+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:14.558 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:14.552+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:14.940 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:14.934+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:15.387 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:15.383+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:15.421 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:15.415+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:15.447 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:15.442+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:15.453 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:15.448+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:15.475 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:15.470+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:15.580 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:15.574+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:15.980 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:15.975+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:16.389 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:24:16.385+0000 16949700 -1 osd.6 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.031545+0000 front 2019-11-25T17:23:21.165643+0000 (oldest deadline 2019-11-25T17:24:01.002679+0000) 2019-11-25T17:24:16.414 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:16.409+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:16.446 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:16.439+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:16.484 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:16.474+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:16.520 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:16.515+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:16.625 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:16.619+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:16.944 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:16.938+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:17.393 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:17.388+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:17.433 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:17.427+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:17.523 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:17.519+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:17.530 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:17.523+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:17.641 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:17.635+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:17.957 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:17.950+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:18.394 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:18.388+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:18.418 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:18.413+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:18.486 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:18.480+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:18.494 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:18.488+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:18.660 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:18.653+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:18.947 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:18.941+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:19.397 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:19.391+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:19.437 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:19.426+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:19.508 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:19.502+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:19.530 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:19.524+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:19.626 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:19.619+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:19.933 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:19.926+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:20.396 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:20.389+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:20.411 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:20.406+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:20.486 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:20.479+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:20.577 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:20.571+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:20.679 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:20.673+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:20.919 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:20.912+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:21.391 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:21.384+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:21.464 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:21.460+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:21.525 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:21.518+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:21.564 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:21.558+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:21.636 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:21.630+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:21.875 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:21.868+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:22.422 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:22.417+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:22.428 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:22.422+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:22.480 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:22.473+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:22.548 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:22.543+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:22.653 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:22.647+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:22.916 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:22.910+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:23.389 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:23.383+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:23.456 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:23.451+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:23.541 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:23.531+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:23.562 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:23.557+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:23.703 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:23.698+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:23.936 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:23.929+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:24.344 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:24.337+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:24.474 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:24.468+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:24.518 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:24.512+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:24.526 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:24.519+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:24.691 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:24.685+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:24.900 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:24.890+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:25.397 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:25.391+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:25.481 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:25.475+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:25.500 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:25.495+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:25.720 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:25.621+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:25.724 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:25.719+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:25.934 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:25.929+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:26.382 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:26.376+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:26.458 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:26.451+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:26.513 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:26.508+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:26.675 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:26.670+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:26.772 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:26.766+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:26.964 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:26.958+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:27.362 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:27.356+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:27.481 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:27.475+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:27.499 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:27.493+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:27.684 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:27.678+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:27.732 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:27.727+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:27.956 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:27.951+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:28.339 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:28.333+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:28.492 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:28.486+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:28.534 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:28.528+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:28.654 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:28.647+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:28.741 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:28.736+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:28.945 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:28.940+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:29.337 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:29.331+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:29.487 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:29.482+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:29.506 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:29.499+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:29.705 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:29.698+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:29.709 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:29.703+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:29.944 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:29.938+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:30.324 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:30.318+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:30.496 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:30.490+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:30.514 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:30.508+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:30.660 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:30.655+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:30.729 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:30.723+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:30.930 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:30.923+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:31.330 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:31.325+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:31.532 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:31.525+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:31.548 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:31.542+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:31.701 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:31.696+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:31.710 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:31.706+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:31.889 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:24:31.884+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 40sec e471 v471) 2019-11-25T17:24:31.899 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:31.894+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:32.334 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:32.329+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:32.551 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:32.546+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:32.559 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:32.553+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:32.702 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:32.698+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:32.718 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:32.713+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:32.936 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:32.928+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:33.302 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:33.296+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:33.525 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:33.520+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:33.600 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:33.594+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:33.689 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:33.684+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:33.741 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:33.735+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:33.960 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:33.954+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:34.282 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:34.276+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:34.556 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:34.550+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:34.563 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:34.557+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:34.745 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:34.739+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:34.794 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:34.751+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:34.971 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:34.967+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:35.307 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:35.301+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:35.572 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:35.567+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:35.588 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:35.582+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:35.735 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:35.730+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:35.875 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:35.869+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:36.009 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:36.004+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:36.316 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:36.310+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:36.585 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:36.578+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:36.615 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:36.610+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:36.718 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:36.712+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:36.899 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:36.889+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:36.906 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:24:36.902+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 5 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 40sec e471 v471) 2019-11-25T17:24:36.988 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:36.982+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:37.297 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:37.290+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:37.593 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:37.586+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:37.611 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:37.606+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:37.744 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:37.738+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:37.926 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:37.920+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:38.034 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:38.029+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:38.285 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:38.278+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:38.400 INFO:teuthology.orchestra.run.smithi153.stderr:testMetadataOnPost (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:24:38.615 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:38.609+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:38.659 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:38.654+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:38.784 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:38.779+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:38.984 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:38.971+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:39.019 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:39.015+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:39.241 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:39.235+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:39.621 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:39.614+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:39.760 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:39.711+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:39.795 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:39.791+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:40.003 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:39.997+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:40.032 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:40.027+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:40.197 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:40.190+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:40.639 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:40.634+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:40.767 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:40.761+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:40.796 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:40.791+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:40.967 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:40.961+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:41.066 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:24:41.060+0000 16949700 -1 osd.0 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:20.498841+0000 front 2019-11-25T17:23:22.405121+0000 (oldest deadline 2019-11-25T17:24:03.685650+0000) 2019-11-25T17:24:41.246 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:24:41.240+0000 16949700 -1 osd.7 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.265332+0000 front 2019-11-25T17:23:19.017453+0000 (oldest deadline 2019-11-25T17:24:04.220639+0000) 2019-11-25T17:24:41.680 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:24:41.675+0000 16949700 -1 osd.5 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:19.287782+0000 front 2019-11-25T17:23:19.311237+0000 (oldest deadline 2019-11-25T17:24:04.843137+0000) 2019-11-25T17:24:41.744 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:24:41.739+0000 16949700 -1 osd.3 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.461090+0000 front 2019-11-25T17:23:21.130042+0000 (oldest deadline 2019-11-25T17:24:01.094632+0000) 2019-11-25T17:24:41.800 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:24:41.795+0000 16949700 -1 osd.1 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:17.438280+0000 front 2019-11-25T17:23:18.991139+0000 (oldest deadline 2019-11-25T17:24:01.991844+0000) 2019-11-25T17:24:41.967 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:24:41.960+0000 16949700 -1 osd.2 471 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:23:21.211541+0000 front 2019-11-25T17:23:21.186655+0000 (oldest deadline 2019-11-25T17:24:02.269150+0000) 2019-11-25T17:24:41.972 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:24:41.964+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 40sec e471 v471) 2019-11-25T17:25:15.653 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:25:15.612+0000 36272700 -1 osd.3 475 build_incremental_map_msg missing incremental map 475 2019-11-25T17:25:15.760 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:25:15.688+0000 34a6f700 -1 osd.3 475 build_incremental_map_msg missing incremental map 475 2019-11-25T17:25:15.936 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:25:15.765+0000 35a71700 -1 osd.3 475 build_incremental_map_msg missing incremental map 475 2019-11-25T17:25:16.145 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:25:15.968+0000 35270700 -1 osd.3 475 build_incremental_map_msg missing incremental map 475 2019-11-25T17:25:17.748 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:25:17.710+0000 3aa7b700 -1 osd.3 477 build_incremental_map_msg missing incremental map 475 2019-11-25T17:25:26.443 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:10:31.026 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:26.450 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:10:31.032 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:26.451 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:10:31.057 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:26.454 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:10:31.060 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:26.456 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:10:31.061 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:26.459 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:10:31.064 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:26.542 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:10:31.125 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:26.551 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:10:31.156 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:26.579 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:10:31.184 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:25:47.198 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:47.193+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:48.238 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:48.232+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:49.227 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:49.221+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:50.273 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:50.267+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:51.313 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:51.308+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:52.368 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:52.362+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:53.393 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:53.388+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:54.352 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:54.347+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:55.383 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:55.379+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:56.400 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:56.395+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:57.426 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:57.422+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:58.461 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:58.456+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:25:59.513 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:25:59.508+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:00.488 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:00.483+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:01.481 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:01.476+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:02.460 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:02.455+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:03.447 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:03.443+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:04.423 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:04.418+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:05.416 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:05.411+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:06.468 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:06.462+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:07.431 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:07.425+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:08.396 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:08.388+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:09.358 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:09.353+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:10.382 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:10.376+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:11.373 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:11.368+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:12.349 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:12.344+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:13.386 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:13.380+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:14.400 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:14.392+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:15.363 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:15.357+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:16.415 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:16.410+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:17.452 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:17.447+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:18.496 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:18.491+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:19.504 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:19.499+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:20.511 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:20.505+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:20.903 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:20.898+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:21.547 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:21.541+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:21.929 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:21.924+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:22.535 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:22.530+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:22.953 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:22.948+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:23.258 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:23.253+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:23.544 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:23.538+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:23.620 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:23.614+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:23.969 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:23.964+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:23.991 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:23.985+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:24.258 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:24.252+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:24.503 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:24.498+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:24.649 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:24.644+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:24.944 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:24.939+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:24.994 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:24.987+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:25.289 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:25.283+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:25.516 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:25.511+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:25.671 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:25.665+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:25.909 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:25.905+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:26.020 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:26.014+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:26.250 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:26.243+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:26.488 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:26.481+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:26.489 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:26.483+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:26.682 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:26.676+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:26.880 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:26.875+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:26.889 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:26.884+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:27.037 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:27.031+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:27.254 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:27.248+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:27.490 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:27.484+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:27.490 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:27.485+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:27.691 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:27.686+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:27.742 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:27.736+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:27.916 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:27.910+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:27.920 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:27.915+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:28.076 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:28.070+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:28.218 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:28.213+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:28.494 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:28.486+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:28.494 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:28.488+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:28.724 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:28.719+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:28.762 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:28.756+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:28.882 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:28.877+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:28.951 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:28.945+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:29.059 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:29.054+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:29.240 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:29.234+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:29.536 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:29.530+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:29.539 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:29.534+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:29.774 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:29.769+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:29.815 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:29.809+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:29.848 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:29.842+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:29.965 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:29.960+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:30.068 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:30.062+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:30.232 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:30.227+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:30.505 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:30.500+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:30.508 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:30.503+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:30.809 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:30.803+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:30.869 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:30.863+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:30.898 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:30.893+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:31.021 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:31.012+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:31.045 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:31.039+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:31.217 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:31.211+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:31.559 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:31.554+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:31.561 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:31.556+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:31.857 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:31.851+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:31.900 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:31.890+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:31.950 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:31.945+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:32.061 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:32.055+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:32.082 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:32.077+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:32.247 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:32.241+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:32.526 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:32.520+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:32.527 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:32.521+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:32.863 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:32.859+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:32.867 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:32.860+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:32.904 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:32.899+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:33.069 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:33.064+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:33.129 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:33.123+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:33.294 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:33.287+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:33.482 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:33.475+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:33.483 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:33.477+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:33.829 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:33.823+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:33.833 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:33.827+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:33.861 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:33.856+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:34.129 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:34.116+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:34.174 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:34.168+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:34.291 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:34.285+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:34.521 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:34.514+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:34.522 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:34.516+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:34.819 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:34.814+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:34.863 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:34.858+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:34.869 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:34.864+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:35.154 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:35.148+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:35.155 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:35.150+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:35.301 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:35.294+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:35.558 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:35.551+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:35.558 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:35.553+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:35.780 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:35.775+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:35.909 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:35.903+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:35.918 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:35.913+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:36.126 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:36.120+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:36.201 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:36.195+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:36.259 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:36.253+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:36.585 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:36.579+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:36.585 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:36.580+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:36.823 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:36.819+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:36.876 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:36.870+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:36.928 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:36.924+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:37.119 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:37.114+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:37.184 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:37.178+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:37.236 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:37.230+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:37.561 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:37.555+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:37.561 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:37.556+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:37.794 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:37.788+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:37.866 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:37.860+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:37.910 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:37.904+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:38.080 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:38.074+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:38.190 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:38.184+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:38.269 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:38.262+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:38.550 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:38.545+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:38.553 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:38.548+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:38.771 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:38.767+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:38.835 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:38.829+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:38.907 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:38.901+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:39.072 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:39.067+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:39.186 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:39.180+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:39.242 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:39.235+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:39.607 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:39.599+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:39.608 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:39.601+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:39.757 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:39.751+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:39.862 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:39.856+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:39.877 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:39.871+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:40.030 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:40.024+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:40.184 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:40.178+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:40.219 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:40.213+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:40.621 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:40.613+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:40.621 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:40.615+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:40.768 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:40.763+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:40.828 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:40.822+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:40.840 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:40.835+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:41.012 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:41.007+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:41.202 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:41.197+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:41.228 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:41.223+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:41.654 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:41.647+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:41.655 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:41.649+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:41.789 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:41.784+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:41.790 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:41.785+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:41.880 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:41.875+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:42.009 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:42.005+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:42.198 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:42.192+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:42.200 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:42.195+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:42.619 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:42.613+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:42.620 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:42.614+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:42.759 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:42.754+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:42.781 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:42.776+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:42.871 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:42.865+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:43.005 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:42.999+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:43.176 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:43.170+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:43.202 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:43.197+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:43.643 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:43.637+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:43.646 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:43.641+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:43.784 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:43.779+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:43.804 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:43.798+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:43.918 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:43.913+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:44.011 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:44.006+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:44.151 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:44.146+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:44.182 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:44.177+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:44.667 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:44.661+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:44.668 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:44.663+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:44.763 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:44.758+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:44.830 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:44.824+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:44.898 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:44.893+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:45.038 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:45.021+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:45.192 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:45.188+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:45.205 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:45.199+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:45.663 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:45.657+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:45.663 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:45.659+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:45.737 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:45.732+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:45.783 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:45.778+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:45.906 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:45.900+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:46.057 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:46.051+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:46.194 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:46.190+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:46.214 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:46.209+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:46.654 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:46.648+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:46.655 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:46.649+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:46.769 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:46.763+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:46.772 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:46.766+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:46.909 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:46.903+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:47.114 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:47.108+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:47.193 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:47.183+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:47.262 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:47.257+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:47.702 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:47.696+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:47.703 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:47.698+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:47.745 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:47.739+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:47.758 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:47.753+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:47.891 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:47.879+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:48.073 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:48.068+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:48.201 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:48.196+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:48.315 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:48.310+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:48.710 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:48.703+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:48.711 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:48.705+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:48.736 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:48.730+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:48.738 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:48.734+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:48.839 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:48.834+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:49.033 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:49.028+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:49.211 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:49.206+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:49.285 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:49.279+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:49.714 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:49.710+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:49.715 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:49.711+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:49.728 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:49.723+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:49.729 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:49.723+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:49.844 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:49.839+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:50.072 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:50.066+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:50.253 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:50.249+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:50.331 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:50.326+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:50.694 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:50.690+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:50.694 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:50.690+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:50.738 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:50.732+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:50.739 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:50.733+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:50.863 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:50.858+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:51.086 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:51.080+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:51.214 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:51.209+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:51.355 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:51.350+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:51.698 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:51.692+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:51.714 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:51.709+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:51.717 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:51.713+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:51.723 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:51.718+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:51.880 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:51.875+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:52.065 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:52.061+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:52.251 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:52.246+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:52.381 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:52.376+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:52.716 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:52.711+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:52.717 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:52.712+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:52.741 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:52.737+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:52.780 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:52.775+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:52.865 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:52.861+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:53.097 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:53.092+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:53.271 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:53.265+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:53.358 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:53.352+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:53.698 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:53.694+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:53.701 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:53.697+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:53.709 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:53.703+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:53.809 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:53.804+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:53.912 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:53.906+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:54.127 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:54.123+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:54.271 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:54.265+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:54.348 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:54.343+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:54.694 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:54.689+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:54.696 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:54.692+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:54.763 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:54.756+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:54.769 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:54.765+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:54.939 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:54.933+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:55.162 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:55.156+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:55.290 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:55.285+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:55.369 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:55.364+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:55.731 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:55.727+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:55.732 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:55.729+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:55.736 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:55.731+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:55.759 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:55.754+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:55.940 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:26:55.934+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.177306+0000 front 2019-11-25T17:25:44.964163+0000 (oldest deadline 2019-11-25T17:26:22.916774+0000) 2019-11-25T17:26:56.212 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:56.198+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:56.294 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:26:56.290+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.065665+0000 front 2019-11-25T17:25:34.583621+0000 (oldest deadline 2019-11-25T17:26:20.441942+0000) 2019-11-25T17:26:56.345 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:56.340+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:56.726 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:56.720+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:56.767 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:26:56.762+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.064871+0000 front 2019-11-25T17:25:41.063330+0000 (oldest deadline 2019-11-25T17:26:23.344456+0000) 2019-11-25T17:26:56.773 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:56.769+0000 16949700 -1 osd.4 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:43.167844+0000 front 2019-11-25T17:25:44.315479+0000 (oldest deadline 2019-11-25T17:26:25.805576+0000) 2019-11-25T17:26:56.779 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:56.775+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:57.191 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:57.186+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:57.393 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:57.383+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:57.566 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:26:57.557+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.6 [v2:172.21.15.81:6816/9710,v1:172.21.15.81:6817/9710] for 40sec e477 v477) 2019-11-25T17:26:57.770 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:26:57.765+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:41.164948+0000 front 2019-11-25T17:25:44.677550+0000 (oldest deadline 2019-11-25T17:26:26.972490+0000) 2019-11-25T17:26:58.179 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:58.172+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:58.344 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:57.999+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=51 ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:26:58.360 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:58.355+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:59.163 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:26:59.157+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.699534+0000 front 2019-11-25T17:25:44.686592+0000 (oldest deadline 2019-11-25T17:26:26.121077+0000) 2019-11-25T17:26:59.363 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:26:59.352+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:25:44.183427+0000 front 2019-11-25T17:25:44.962536+0000 (oldest deadline 2019-11-25T17:26:22.464378+0000) 2019-11-25T17:26:59.733 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:26:59.393+0000 16949700 -1 osd.4 477 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=51 ondisk+retry+read+known_if_redirected e475) 2019-11-25T17:27:12.470 INFO:teuthology.orchestra.run.smithi153.stderr:testMetadataOnPut (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:27:24.915 INFO:teuthology.orchestra.run.smithi153.stderr:testNameLimit (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:27:24.962 INFO:teuthology.orchestra.run.smithi153.stderr:testNoContentLengthForPut (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:27:53.612 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:27:53.606+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:27:53.713 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:27:53.706+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:27:54.573 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:27:54.566+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:27:54.753 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:27:54.748+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:27:54.948 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:54.941+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:27:55.311 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:27:55.304+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:27:55.615 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:27:55.608+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:27:55.752 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:27:55.746+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:27:55.779 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:27:55.772+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:27:55.990 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:55.985+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:27:55.994 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:55.990+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:27:56.307 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:27:56.300+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:27:56.585 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:27:56.579+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:27:56.739 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:27:56.733+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:27:56.759 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:27:56.752+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:27:56.770 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:27:56.762+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:27:57.012 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:57.007+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:27:57.012 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:57.008+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:27:57.307 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:27:57.300+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:27:57.334 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:27:57.328+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:27:57.586 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:27:57.580+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:27:57.750 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:27:57.743+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:27:57.769 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:27:57.763+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:27:57.795 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:27:57.789+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:27:57.968 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:57.963+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:27:57.970 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:57.966+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:27:58.286 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:27:58.281+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:27:58.340 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:27:58.334+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:27:58.597 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:27:58.591+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:27:58.739 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:27:58.733+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:27:58.777 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:27:58.767+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:27:58.795 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:27:58.789+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:27:58.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:58.986+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:27:58.992 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:58.988+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:27:59.296 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:27:59.290+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:27:59.334 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:27:59.329+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:27:59.628 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:27:59.622+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:27:59.708 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:27:59.701+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:27:59.775 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:27:59.768+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:27:59.837 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:27:59.830+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:27:59.979 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:59.973+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:27:59.979 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:27:59.974+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:00.277 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:00.272+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:00.331 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:00.322+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:00.678 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:00.670+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:00.692 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:00.686+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:00.741 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:00.735+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:00.798 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:00.793+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:00.960 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:00.954+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:00.964 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:00.958+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:01.258 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:01.253+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:01.304 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:01.299+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:01.658 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:01.652+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:01.702 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:01.697+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:01.741 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:01.735+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:01.828 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:01.823+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:01.921 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:01.915+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:01.924 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:01.919+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:02.229 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:02.224+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:02.267 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:02.263+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:02.659 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:02.653+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:02.701 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:02.694+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:02.708 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:02.702+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:02.826 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:02.820+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:02.940 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:02.935+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:02.941 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:02.936+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:03.281 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:03.277+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:03.313 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:03.307+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:03.669 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:03.662+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:03.680 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:03.674+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:03.682 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:03.675+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:03.788 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:03.783+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:03.937 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:03.931+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:03.938 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:03.933+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:04.297 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:04.291+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:04.353 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:04.347+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:04.665 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:04.658+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:04.690 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:04.684+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:04.719 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:04.713+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:04.778 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:04.773+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:04.922 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:04.917+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:04.926 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:04.921+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:05.323 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:05.317+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:05.334 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:05.330+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:05.645 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:05.639+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:05.685 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:05.678+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:05.734 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:05.728+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:05.809 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:05.804+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:05.933 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:05.927+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:05.934 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:05.929+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:06.332 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:06.326+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:06.365 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:06.358+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:06.665 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:06.659+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:06.682 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:06.676+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:06.733 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:06.727+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:06.777 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:06.770+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:06.923 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:06.917+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:06.924 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:06.919+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:07.370 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:07.364+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:07.406 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:07.401+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:07.660 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:07.653+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:07.706 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:07.700+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:07.720 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:07.714+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:07.769 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:07.764+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:07.890 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:07.883+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:07.890 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:07.886+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:08.339 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:08.333+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:08.374 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:08.369+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:08.676 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:08.670+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:08.699 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:08.693+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:08.759 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:08.752+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:08.768 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:08.762+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:08.852 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:08.847+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:08.854 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:08.850+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:09.353 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:09.348+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:09.395 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:09.390+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:09.653 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:09.648+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:09.675 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:09.669+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:09.726 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:09.721+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:09.806 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:09.800+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:09.898 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:09.891+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:09.898 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:09.893+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:10.368 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:10.362+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:10.398 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:10.392+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:10.655 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:10.649+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:10.682 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:10.676+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:10.697 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:10.691+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:10.843 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:10.836+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:10.896 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:10.891+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:10.898 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:10.894+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:11.347 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:11.343+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:11.444 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:11.438+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:11.621 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:11.616+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:11.702 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:11.696+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:11.718 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:11.711+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:11.865 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:11.858+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:11.949 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:11.943+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:11.950 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:11.945+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:12.347 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:12.341+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:12.407 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:12.401+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:12.665 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:12.659+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:12.734 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:12.729+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:12.745 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:12.739+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:12.873 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:12.867+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:12.948 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:12.942+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:12.949 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:12.944+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:13.322 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:13.317+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:13.389 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:13.378+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:13.628 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:13.622+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:13.733 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:13.727+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:13.754 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:13.747+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:13.835 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:13.829+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:13.986 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:13.980+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:13.987 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:13.982+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:14.289 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:14.283+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:14.359 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:14.354+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:14.670 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:14.664+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:14.724 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:14.715+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:14.741 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:14.735+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:14.864 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:14.858+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:14.964 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:14.957+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:14.965 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:14.960+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:15.266 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:15.261+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:15.385 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:15.380+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:15.697 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:15.690+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:15.699 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:15.693+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:15.718 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:15.711+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:15.833 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:15.827+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:15.998 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:15.992+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:16.000 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:15.996+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:16.226 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:16.221+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:16.414 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:16.408+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:16.714 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:16.708+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:16.721 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:16.717+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:16.765 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:16.758+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:16.874 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:16.867+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:16.966 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:16.961+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:16.967 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:16.962+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:17.198 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:17.192+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:17.458 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:17.452+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:17.711 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:17.704+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:17.732 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:17.726+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:17.772 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:17.767+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:17.875 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:17.868+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:17.943 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:17.937+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:17.944 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:17.939+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:18.203 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:18.197+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:18.459 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:18.453+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:18.713 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:18.706+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:18.777 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:18.771+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:18.798 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:18.793+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:18.875 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:18.869+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:18.945 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:18.939+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:18.948 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:18.943+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:19.198 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:19.192+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:19.450 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:19.445+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:19.674 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:19.669+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:19.813 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:19.807+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:19.824 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:19.818+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:19.903 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:19.897+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:19.954 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:19.949+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:19.955 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:19.950+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:20.167 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:20.161+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:20.483 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:20.477+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:20.682 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:20.675+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:20.831 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:20.825+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:20.833 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:20.828+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:20.872 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:20.865+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:20.957 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:20.951+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:20.958 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:20.952+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:21.186 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:21.181+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:21.444 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:21.439+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:21.725 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:21.719+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:21.848 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:21.842+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:21.854 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:21.848+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:21.871 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:21.864+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:21.954 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:21.948+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:21.955 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:21.950+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:22.177 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:22.171+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:22.475 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:22.470+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:22.711 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:22.705+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:22.828 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:22.821+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:22.846 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:22.840+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:22.886 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:22.880+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:22.928 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:22.922+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:22.928 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:22.923+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:23.167 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:23.161+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:23.451 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:23.445+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:23.667 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:23.661+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:23.827 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:23.820+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:23.846 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:23.840+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:23.869 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:23.864+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:23.890 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:23.876+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:23.890 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:23.877+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:24.213 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:24.208+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:24.471 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:24.465+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:24.698 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:24.691+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:24.787 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:24.781+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:24.839 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:24.832+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:24.848 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:24.842+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:24.884 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:24.878+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:24.884 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:24.880+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:25.233 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:25.227+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:25.478 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:25.471+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:25.718 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:25.711+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:25.832 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:25.826+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:25.833 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:25.827+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:25.850 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:25.844+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:25.922 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:25.916+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:25.922 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:25.917+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:26.243 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:26.237+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:26.439 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:26.434+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:26.717 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:26.712+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:26.788 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:26.782+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:26.811 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:26.806+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:26.865 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:26.859+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:26.919 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:26.914+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:26.920 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:26.915+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:27.296 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:27.290+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:27.439 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:27.433+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:27.729 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:27.723+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:27.769 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:27.763+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:27.851 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:27.845+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:27.852 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:27.847+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:27.861 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:28:27.857+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 4 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 43sec e477 v477) 2019-11-25T17:28:27.887 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:27.882+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:27.889 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:27.885+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:28.299 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:28.294+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:28.445 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:28.439+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:28.778 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:28.772+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:28.779 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:28.775+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:28.850 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:28.845+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:28.876 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:28.871+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:28.929 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:28.923+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:28.929 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:28.925+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:29.303 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:29.297+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:29.479 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:29.473+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:29.813 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:29.807+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:29.822 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:29.816+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:29.883 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:29.877+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:29.911 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:29.905+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:29.914 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:29.907+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:29.914 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:29.909+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:30.333 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:30.328+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:30.508 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:30.498+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:30.792 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:30.786+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:30.844 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:30.839+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:30.898 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:30.893+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:30.900 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:30.894+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:30.901 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:30.896+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:30.951 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:30.944+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:31.379 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:31.373+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:31.478 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:31.471+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:31.773 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:31.769+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:31.808 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:31.803+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:31.938 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:31.931+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:31.939 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:31.933+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:31.939 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:31.934+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:31.974 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:31.968+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:32.371 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:28:32.365+0000 16949700 -1 osd.2 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.060082+0000 front 2019-11-25T17:27:12.066083+0000 (oldest deadline 2019-11-25T17:27:56.844016+0000) 2019-11-25T17:28:32.493 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:28:32.487+0000 16949700 -1 osd.1 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.379700+0000 front 2019-11-25T17:27:12.360956+0000 (oldest deadline 2019-11-25T17:27:54.725296+0000) 2019-11-25T17:28:32.821 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:28:32.816+0000 16949700 -1 osd.6 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.286638+0000 front 2019-11-25T17:27:12.284885+0000 (oldest deadline 2019-11-25T17:27:53.688696+0000) 2019-11-25T17:28:32.828 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:28:32.824+0000 16949700 -1 osd.5 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.273867+0000 front 2019-11-25T17:27:12.058416+0000 (oldest deadline 2019-11-25T17:27:56.035608+0000) 2019-11-25T17:28:32.929 INFO:tasks.ceph.mon.b.smithi081.stderr:2019-11-25T17:28:32.921+0000 1a898700 -1 mon.b@0(leader) e1 get_health_metrics reporting 7 slow ops, oldest is osd_failure(failed timeout osd.4 [v2:172.21.15.81:6824/9716,v1:172.21.15.81:6825/9716] for 43sec e477 v477) 2019-11-25T17:28:32.933 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:32.927+0000 16949700 -1 osd.3 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:11.271783+0000 front 2019-11-25T17:27:11.270360+0000 (oldest deadline 2019-11-25T17:27:54.557553+0000) 2019-11-25T17:28:32.934 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:32.929+0000 16949700 -1 osd.3 477 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] ondisk+read+known_if_redirected e477) 2019-11-25T17:28:32.957 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:28:32.954+0000 16949700 -1 osd.7 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:12.126510+0000 front 2019-11-25T17:27:12.093656+0000 (oldest deadline 2019-11-25T17:27:53.480088+0000) 2019-11-25T17:28:32.975 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:28:32.970+0000 16949700 -1 osd.0 477 heartbeat_check: no reply from 172.21.15.81:6828 osd.4 since back 2019-11-25T17:27:13.051529+0000 front 2019-11-25T17:27:12.919092+0000 (oldest deadline 2019-11-25T17:27:55.277329+0000) 2019-11-25T17:28:36.019 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:36.014+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:37.041 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:37.037+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:38.089 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:38.085+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:39.074 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:39.070+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:40.065 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:40.060+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:41.051 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:41.046+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:42.097 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:42.092+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:43.089 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:43.085+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:44.121 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:44.115+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:45.107 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:45.101+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:46.062 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:46.056+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:47.055 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:47.049+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:48.010 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:48.004+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:48.972 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:48.963+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:50.010 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:50.005+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:51.039 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:51.034+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:52.049 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:52.045+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:53.007 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:53.002+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:54.020 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:54.015+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:54.999 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:54.994+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:56.035 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:56.030+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:57.033 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:57.028+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:57.993 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:57.988+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:59.025 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:59.021+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:28:59.991 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:28:59.986+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:29:01.046 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:29:01.042+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:29:02.062 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:29:02.057+0000 16949700 -1 osd.3 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124261 6.2 6:43664ecc:root::%e0%a4%82%e0%a4%80%e5%98%87%ed%88%8f%e0%a4%82%e9%90%8b%eb%8c%8d%e9%90%8a%e1%a0%83%e9%90%89%eb%8c%8d%e2%98%83%eb%8c%8e%e1%a0%82%e9%90%8a:head [call version.read in=11b,getxattrs,stat] snapc 0=[] RETRY=1 ondisk+retry+read+known_if_redirected e477) 2019-11-25T17:29:02.920 INFO:teuthology.orchestra.run.smithi153.stderr:testPostOnFileThatDoesNotExist (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:29:03.469 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:29:03.353+0000 16949700 -1 osd.6 480 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=52 ondisk+retry+read+known_if_redirected e478) 2019-11-25T17:29:03.593 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:29:03.588+0000 16949700 -1 osd.1 479 get_health_metrics reporting 1 slow ops, oldest is osd_op(client.4500.0:124320 6.9 6:999bad5f:users.uid::foo.client.0.buckets:head [call user.set_buckets_info in=249b] snapc 0=[] RETRY=1 ondisk+retry+write+known_if_redirected e478) 2019-11-25T17:29:04.655 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:29:04.493+0000 16949700 -1 osd.6 480 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=52 ondisk+retry+read+known_if_redirected e478) 2019-11-25T17:29:05.767 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:29:05.638+0000 16949700 -1 osd.6 481 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=52 ondisk+retry+read+known_if_redirected e478) 2019-11-25T17:29:06.595 INFO:tasks.ceph.osd.4.smithi081.stderr:==00:08:13:48.341 9716== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:06.732 INFO:tasks.ceph.osd.4.smithi081.stderr:==00:08:13:48.478 9716== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:06.853 INFO:tasks.ceph.osd.6.smithi081.stderr:2019-11-25T17:29:06.837+0000 16949700 -1 osd.6 481 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7:5127e642:gc::gc.21:head [call rgw_gc.rgw_gc_queue_list_entries in=46b] snapc 0=[] RETRY=52 ondisk+retry+read+known_if_redirected e478) 2019-11-25T17:29:31.477 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:14:36.059 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:31.484 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:14:36.066 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:31.497 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:14:36.103 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:31.502 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:14:36.107 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:31.504 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:14:36.110 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:31.506 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:14:36.111 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:31.714 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:14:36.296 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:31.780 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:14:36.385 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:31.822 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:14:36.428 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:38.132 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:38.126+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:39.151 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:39.146+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:40.156 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:40.151+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:41.142 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:41.136+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:42.120 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:42.113+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:43.119 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:43.114+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:44.112 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:44.107+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:45.116 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:45.110+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:46.128 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:46.123+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:47.147 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:47.141+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:47.553 INFO:teuthology.orchestra.run.smithi153.stderr:testRangedGets (test.functional.tests.TestFileUTF8) ... ok 2019-11-25T17:29:48.164 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:48.159+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:48.387 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:14:52.970 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:48.391 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:14:52.973 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:48.400 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:14:53.005 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:48.404 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:14:53.009 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:48.405 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:14:53.010 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:48.407 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:14:53.013 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:48.439 INFO:tasks.ceph.mon.a.smithi153.stderr:==00:08:14:53.045 8739== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:48.441 INFO:tasks.ceph.mon.c.smithi153.stderr:==00:08:14:53.046 8741== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:48.445 INFO:tasks.ceph.mon.b.smithi081.stderr:==00:08:14:53.028 8379== Warning: unimplemented fcntl command: 1036 2019-11-25T17:29:49.163 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:49.157+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:50.171 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:50.166+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:51.148 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:51.142+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:52.183 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:52.178+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:53.187 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:53.180+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:54.148 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:54.141+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:55.112 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:55.107+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:56.154 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:56.147+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:56.825 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:29:56.820+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:29:57.175 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:57.170+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:57.389 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:29:57.383+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:29:57.698 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:29:57.692+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:29:57.824 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:29:57.819+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:29:58.229 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:58.220+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:58.410 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:29:58.405+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:29:58.686 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:29:58.680+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:29:58.844 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:29:58.838+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:29:59.204 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:59.198+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:29:59.204 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:29:59.199+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:29:59.419 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:29:59.414+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:29:59.712 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:29:59.706+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:29:59.856 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:29:59.852+0000 16949700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:16.145347+0000 front 2019-11-25T17:29:15.855992+0000 (oldest deadline 2019-11-25T17:29:59.609269+0000) 2019-11-25T17:29:59.878 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:29:59.856+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:30:00.167 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:00.160+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:30:00.168 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:00.161+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:30:00.438 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:30:00.432+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:30:00.757 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:30:00.751+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:30:00.851 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:30:00.844+0000 16949700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:15.502153+0000 front 2019-11-25T17:29:15.274388+0000 (oldest deadline 2019-11-25T17:30:00.835459+0000) 2019-11-25T17:30:00.856 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:30:00.850+0000 16949700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:16.145347+0000 front 2019-11-25T17:29:15.855992+0000 (oldest deadline 2019-11-25T17:29:59.609269+0000) 2019-11-25T17:30:00.914 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:30:00.909+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:30:01.188 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:01.182+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:30:01.190 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:01.184+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:30:01.205 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:30:01.200+0000 16949700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:14.901767+0000 front 2019-11-25T17:29:14.900665+0000 (oldest deadline 2019-11-25T17:30:00.712679+0000) 2019-11-25T17:30:01.413 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:30:01.406+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:30:01.773 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:30:01.767+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:30:01.850 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:30:01.844+0000 16949700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:15.502153+0000 front 2019-11-25T17:29:15.274388+0000 (oldest deadline 2019-11-25T17:30:00.835459+0000) 2019-11-25T17:30:01.889 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:30:01.884+0000 16949700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:16.145347+0000 front 2019-11-25T17:29:15.855992+0000 (oldest deadline 2019-11-25T17:29:59.609269+0000) 2019-11-25T17:30:01.918 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:30:01.913+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:30:02.153 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:02.146+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:30:02.153 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:02.148+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:30:02.229 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:30:02.223+0000 16949700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:14.901767+0000 front 2019-11-25T17:29:14.900665+0000 (oldest deadline 2019-11-25T17:30:00.712679+0000) 2019-11-25T17:30:02.430 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:30:02.425+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:30:02.729 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:30:02.723+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:30:02.885 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:30:02.879+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:30:02.885 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:30:02.878+0000 16949700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:15.502153+0000 front 2019-11-25T17:29:15.274388+0000 (oldest deadline 2019-11-25T17:30:00.835459+0000) 2019-11-25T17:30:02.902 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:30:02.896+0000 16949700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:16.145347+0000 front 2019-11-25T17:29:15.855992+0000 (oldest deadline 2019-11-25T17:29:59.609269+0000) 2019-11-25T17:30:03.144 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:03.137+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:30:03.147 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:03.142+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:30:03.198 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:30:03.194+0000 16949700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:14.901767+0000 front 2019-11-25T17:29:14.900665+0000 (oldest deadline 2019-11-25T17:30:00.712679+0000) 2019-11-25T17:30:03.483 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:30:03.478+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:30:03.687 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:30:03.681+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:30:03.856 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:30:03.851+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:30:03.922 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:30:03.916+0000 16949700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:16.145347+0000 front 2019-11-25T17:29:15.855992+0000 (oldest deadline 2019-11-25T17:29:59.609269+0000) 2019-11-25T17:30:03.930 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:30:03.925+0000 16949700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:15.502153+0000 front 2019-11-25T17:29:15.274388+0000 (oldest deadline 2019-11-25T17:30:00.835459+0000) 2019-11-25T17:30:04.118 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:04.111+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:30:04.118 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:04.113+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:30:04.157 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:30:04.152+0000 16949700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:14.901767+0000 front 2019-11-25T17:29:14.900665+0000 (oldest deadline 2019-11-25T17:30:00.712679+0000) 2019-11-25T17:30:04.477 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:30:04.472+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:30:04.695 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:30:04.688+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:30:04.891 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:30:04.885+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:30:04.910 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:30:04.904+0000 16949700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:16.145347+0000 front 2019-11-25T17:29:15.855992+0000 (oldest deadline 2019-11-25T17:29:59.609269+0000) 2019-11-25T17:30:04.986 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:30:04.980+0000 16949700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:15.502153+0000 front 2019-11-25T17:29:15.274388+0000 (oldest deadline 2019-11-25T17:30:00.835459+0000) 2019-11-25T17:30:05.097 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:05.089+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:30:05.102 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:05.094+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:30:05.119 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:30:05.114+0000 16949700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:14.901767+0000 front 2019-11-25T17:29:14.900665+0000 (oldest deadline 2019-11-25T17:30:00.712679+0000) 2019-11-25T17:30:05.505 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:30:05.501+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:30:05.676 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:30:05.669+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:30:05.845 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:30:05.840+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:30:05.897 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:30:05.892+0000 16949700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:16.145347+0000 front 2019-11-25T17:29:15.855992+0000 (oldest deadline 2019-11-25T17:29:59.609269+0000) 2019-11-25T17:30:05.960 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:30:05.954+0000 16949700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:15.502153+0000 front 2019-11-25T17:29:15.274388+0000 (oldest deadline 2019-11-25T17:30:00.835459+0000) 2019-11-25T17:30:06.096 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:06.090+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:30:06.099 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:06.094+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:30:06.148 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:30:06.143+0000 16949700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:14.901767+0000 front 2019-11-25T17:29:14.900665+0000 (oldest deadline 2019-11-25T17:30:00.712679+0000) 2019-11-25T17:30:06.544 INFO:tasks.ceph.osd.0.smithi153.stderr:2019-11-25T17:30:06.539+0000 16949700 -1 osd.0 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.260549+0000 front 2019-11-25T17:29:13.187811+0000 (oldest deadline 2019-11-25T17:29:57.161849+0000) 2019-11-25T17:30:06.712 INFO:tasks.ceph.osd.5.smithi081.stderr:2019-11-25T17:30:06.707+0000 16949700 -1 osd.5 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:17.741642+0000 front 2019-11-25T17:29:17.281174+0000 (oldest deadline 2019-11-25T17:29:57.234822+0000) 2019-11-25T17:30:06.862 INFO:tasks.ceph.osd.1.smithi153.stderr:2019-11-25T17:30:06.856+0000 16949700 -1 osd.1 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:12.850274+0000 front 2019-11-25T17:29:12.849186+0000 (oldest deadline 2019-11-25T17:29:56.585267+0000) 2019-11-25T17:30:06.915 INFO:tasks.ceph.osd.2.smithi153.stderr:2019-11-25T17:30:06.909+0000 16949700 -1 osd.2 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:16.145347+0000 front 2019-11-25T17:29:15.855992+0000 (oldest deadline 2019-11-25T17:29:59.609269+0000) 2019-11-25T17:30:06.981 INFO:tasks.ceph.osd.7.smithi081.stderr:2019-11-25T17:30:06.975+0000 16949700 -1 osd.7 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:15.502153+0000 front 2019-11-25T17:29:15.274388+0000 (oldest deadline 2019-11-25T17:30:00.835459+0000) 2019-11-25T17:30:07.099 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:07.092+0000 16949700 -1 osd.4 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:13.776276+0000 front 2019-11-25T17:29:13.805807+0000 (oldest deadline 2019-11-25T17:29:58.409283+0000) 2019-11-25T17:30:07.101 INFO:tasks.ceph.osd.4.smithi081.stderr:2019-11-25T17:30:07.096+0000 16949700 -1 osd.4 482 get_health_metrics reporting 4 slow ops, oldest is osd_op(client.4358.0:91197 7.a 7.4267e48a (undecoded) ondisk+retry+read+known_if_redirected e481) 2019-11-25T17:30:07.168 INFO:tasks.ceph.osd.3.smithi153.stderr:2019-11-25T17:30:07.158+0000 16949700 -1 osd.3 482 heartbeat_check: no reply from 172.21.15.81:6820 osd.6 since back 2019-11-25T17:29:14.901767+0000 front 2019-11-25T17:29:14.900665+0000 (oldest deadline 2019-11-25T17:30:00.712679+0000) 2019-11-25T17:30:07.212 DEBUG:teuthology.exit:Got signal 15; running 2 handlers... 2019-11-25T17:30:07.212 DEBUG:teuthology.task.console_log:Killing console logger for smithi153 2019-11-25T17:30:07.213 DEBUG:teuthology.task.console_log:Killing console logger for smithi081 2019-11-25T17:30:07.213 DEBUG:teuthology.task.console_log:Killing console logger for smithi153 2019-11-25T17:30:07.213 DEBUG:teuthology.task.console_log:Killing console logger for smithi081 2019-11-25T17:30:07.214 DEBUG:teuthology.exit:Finished running handlers